Auditory-Visual Speech Processing (AVSP) 2011

Volterra, Italy
September 1-2, 2011

Audiovisual Perception of Counter-Expectational Questions

Joan Borràs-Comes (1), Cecilia Pugliesi (1,2), Pilar Prieto (2,1)

(1) Universitat Pompeu Fabra; (2) ICREA, Barcelona, Spain

The precise nature of the interaction between acoustic and visual information in the perception of prosodic information is a question that still remains unclear. Though the fuzzy logical model of perception (FLMP) has been shown to explain the recognition of segmental information, this model also needs to be tested in the field of suprasegmentals such as facial gestures. The first goal of this paper is to investigate, by means of a computer-generated 3D video character, the interaction between intonational and gestural information in the detection by listeners of counter-expectational questions compared to narrow focus statements. The second goal is to test which specific facial gesture conveys the counter-expectation meaning most clearly. Our results represent a further step for considering an FLMP approach to the analysis of audiovisual prosody.

Index Terms. audiovisual prosody, intonation, facial gestures, eyebrow, models of perception

Full Paper

Bibliographic reference.  Borràs-Comes, Joan / Pugliesi, Cecilia / Prieto, Pilar (2011): "Audiovisual perception of counter-expectational questions", In AVSP-2011, 43-47.