Fourth Workshop on Child, Computer and Interaction (WOCCI 2014)

September 19, 2014

Performance of a Trialogue-Based Prototype System for English Language Assessment for Young Learners

Keelan Evanini, Youngsoon So, Jidong Tao, Diego Zapata-Rivera, Christine Luce, Laura Battistini, Xinhao Wang

Educational Testing Service, Princeton, NJ, USA

This paper describes a trialogue-based system for assessing the spoken language abilities of young learners of English. Specifically, the system employs spoken dialogue system components in interactive, conversation-based assessment tasks involving the test taker and two virtual interlocutors. The tasks are designed to be engaging for young learners of English at the elementary school level by incorporating real-life situations into the conversations and by providing immediate feedback about their spoken responses. The system was deployed in a data collection experiment with 18 young learners of English in a public school in the USA (grades 3-5) from a variety of language backgrounds. The system was able to produce overall task completion scores for each participant that correlated with scores based on human annotations at a rate of r=0:803. In addition, the participants’ responses to a user experience survey indicate that most participants felt that the virtual interlocutors understood their responses. This prototype demonstrates that this approach to using interactive, conversation-based assessments is a viable method of assessing the English language skills of young learners.

Index Terms: spoken dialogue systems, English language assessment, trialogue, young learners

Full Paper

Bibliographic reference.  Evanini, Keelan / So, Youngsoon / Tao, Jidong / Zapata-Rivera, Diego / Luce, Christine / Battistini, Laura / Wang, Xinhao (2014): "Performance of a trialogue-based prototype system for English language assessment for young learners", In WOCCI-2014, 79-84.