Speech Prosody 2004

Nara, Japan
March 23-26, 2004

Audiovisual Perception of Communication Problems

Pashiera Barkhuysen, Emiel Krahmer, Marc Swerts

Communication & Cognition, Tilburg University, The Netherlands

We describe three perception studies in which subjects are offered film fragments (without any dialogue context) of speakers interacting with a spoken dialogue system. In half of these fragments, the speaker is or becomes aware of a communication problem. Subjects have to determine by forced choice which are the problematic fragments. In all three studies, subjects are capable of performing this task to some extent, but with varying levels of correct classifications. We conclude that combining auditory with visual information is beneficial for problem detection.

Full Paper

Bibliographic reference.  Barkhuysen, Pashiera / Krahmer, Emiel / Swerts, Marc (2004): "Audiovisual perception of communication problems", In SP-2004, 271-274.