ISCA International Workshop on Speech and Language Technology in Education (SLaTE 2011)

Venice, Italy
August 24-26, 2011

Evaluating a Text-to-Scene Generation System as an Aid to Literacy

Bob Coyne (1), Cecilia Schudel (1), Michael Bitz (2,3), Julia Hirschberg (1)

(1) Columbia University, New York, NY, USA
(2) Ramapo College of New Jersey, Mahwah, NJ, USA
(3) Center for Educational Pathways, USA

We discuss classroom experiments using WordsEye, a system for automatically generating 3D scenes from English textual descriptions. Input is syntactically and semantically processed to identify a set of graphical objects and constraints which are then rendered as a 3D scene. We describe experiments with the system in a summer literacy enrichment program conducted at the Harlem Educational Activities Fund with 6th grade students, in which students using the system had significantly greater improvement in their literary character and story descriptions in pre- and post- test essays compared with a control. Students reported that using the system helped them imagine the events in the stories they were reading better. We also observed that social interaction engendered by this process was a strong motivator.
Index Terms. text-to-scene generation

Full Paper

Bibliographic reference.  Coyne, Bob / Schudel, Cecilia / Bitz, Michael / Hirschberg, Julia (2011): "Evaluating a text-to-scene generation system as an aid to literacy", In SLaTE-2011, 13-16.