FAAVSP - The 1st Joint Conference on Facial Analysis, Animation, and
Auditory-Visual Speech Processing

Vienna, Austria
September 11-13, 2015

Classification of Auditory-Visual Attitudes in German

Angelika Hönemann (1), Hansjörg Mixdorff (2), Albert Rilliard (3)

(1) University of Bielefeld, Germany
(2) Beuth University Berlin, Germany
(3) LIMSI-CNRS, France

This paper presents results from an auditory-visual recognition experiment employing short utterances of German produced with varying attitudinal expressions. It is based on 16 different kinds of social and/or propositional attitudes which place speakers in various social interactions with a partner of inferior, equal or superior status, and having a communication aim with a positive, neutral or negative, valence. Data from ten German subjects were classified by native perceivers regarding the attitude portrayed. Participants were given five choices: The intended attitude, two closely related attitudes, and two randomly chosen ones. Higher recognition scores were obtained in audio-visual presentations (45%), over 36% with audio-only stimuli. The best recognized attitudes were doubt, (neutral) statement, surprise and irritation which all yielded audio-visual recognition scores over 50%. Lowest recognition scores were obtained for irony, ‘walking-on-eggs’ and politeness. A hierarchical clustering based on correspondence analysis showed that groupings of stimuli in one cluster are consistent with their original labels - these consistent stimuli yield better recognition scores. Conversely, clusters with heterogeneous populations simply aggregate bad performances.

Full Paper

Bibliographic reference.  Hönemann, Angelika / Mixdorff, Hansjörg / Rilliard, Albert (2015): "Classification of auditory-visual attitudes in German", In FAAVSP-2015, 202-207.