Speech Prosody 2008

Campinas, Brazil
May 6-9, 2008

Recognition of Japanese Attitudes in Audio-Visual Speech

Takaaki Shochi (1), Donna Erickson (2), Albert Rilliard (3), Véronique Aubergé (1), Jean-Claude Martin (3)

(1) GIPSA-Lab, Grenoble, France; (2) Showa Music Univ., Kawasaki City, Japan; (3) LIMSI, Orsay, France

The aim of the present work is to investigate how Japanese listeners recognize 12 audio-visual prosodic attitudes of Japanese. Significant influences of the two speakers and three modalities were observed. Generally the audio-visual condition showed the best recognition score, and interesting behavior for audio and visual modality was observed. Attitudes were regrouped into 3 higher-level perceptual categories for the first speaker: polite expressions, the attitudes of "query", and the expressions of imposition of one’s own opinion. The attitude of kyoshuku and surprise are particularly well recognized by visual information.

Full Paper

Bibliographic reference.  Shochi, Takaaki / Erickson, Donna / Rilliard, Albert / Aubergé, Véronique / Martin, Jean-Claude (2008): "Recognition of Japanese attitudes in audio-visual speech", In SP-2008, 689-692.