Ninth International Conference on Spoken Language Processing

Pittsburgh, PA, USA
September 17-21, 2006

Identification of Confusion and Surprise in Spoken Dialog Using Prosodic Features

Rohit Kumar (1), Carolyn P. Rosť (1), Diane J. Litman (2)

(1) Carnegie Mellon University, USA; (2) University of Pittsburgh, USA

Sensitivity to a userís emotional state offers promise in improving the state of the art in spoken dialog systems. In this work, we attempt to detect the speakerís states of confusion and surprise using prosodic features from his/her utterances. We have collected a corpus of utterances in realistic settings using an experimental methodology aimed at eliciting confusion and surprise from users. Classification experiments have yielded up to a 27.2% improvement over baseline performance using F0 and power features. We achieved the greatest success at classification of emotions that were most successfully elicited.

Full Paper

Bibliographic reference.  Kumar, Rohit / Rosť, Carolyn P. / Litman, Diane J. (2006): "Identification of confusion and surprise in spoken dialog using prosodic features", In INTERSPEECH-2006, paper 1921-Wed2BuP.14.