Speech Prosody 2006

Dresden, Germany
May 2-5, 2006

Audio and Audio-visual Effects of a Short English Emotional Sentence on Japanese L2s and English L1s Cognition, and Physio-Acoustic Correlate

Toshiko Isei-Jaakkola (1,3), Qinghua Sun (2), Keikichi Hirose (1)

(1) Graduate School of Information Science and Technology, University of Tokyo, Japan
(2) Graduate School of Engineering, University of Tokyo, Japan
(3) Department of Speech Sciences, University of Helsinki, Finland

The cognition test results of audio (A) and audio-visual (AV) effects on nine English emotions in a short sentence were compared to the physio-acoustic features of sound used for the cognition tests. Two groups of Japanese learners of English (JL2) and one group of English speakers (EL1) participated in these A and AV cognition tests. In the physioacoustic analyses we used F0 and intensity contours and calculated the area of sentential patterns and three forms of distance: area-, average, and pattern-distance for each emotion. Similar patterns of the F0 and intensity contours might have been caused by the cognitive confusions among emotions; the relationships between the cognition tests and physio-acoustic analyses confirmed that there was not a strong correlation between them, intensity seeming to be more correlated to the cognition test results for A for both JL2 and EL1 than F0. EL1s correlation was higher than that of JL2.

Full Paper

Bibliographic reference.  Isei-Jaakkola, Toshiko / Sun, Qinghua / Hirose, Keikichi (2006): "Audio and audio-visual effects of a short English emotional sentence on Japanese L2s and English L1s cognition, and physio-acoustic correlate", In SP-2006, paper 183.