Auditory-Visual Speech Processing (AVSP) 2010

Hakone, Kanagawa, Japan
September 30-October 3, 2010

Cross-Cultural Differences in the Multisensory Perception of Emotion

Akihiro Tanaka (1,4), Ai Koizumi (2), Hisato Imai (3), Saori Hiramatsu (3), Eriko Hiramoto (3), Beatrice de Gelder (4)

(1) Waseda Institute for Advanced Study, Waseda University, Japan
(2) Department of Psychology, The University of Tokyo, Japan
(3) Department of Psychology, Tokyo Woman’s Christian University, Japan
(4) Cognitive and Affective Neuroscience Laboratory, Tilburg University, The Netherlands

Our recent study [1] showed that culture modulates the manner of the multisensory integration of affective information. Specifically, Japanese are more tuned to vocal processing than Dutch in the multisensory perception of emotion. The current study aimed to extend the findings by adding an experiment and conducting further analyses on the results of their study. In the experiments, pairs of affective faces and voices, expressing either congruent or incongruent emotion, were presented simultaneously. In Experiment 1, although the performance on the facial judgment was very high, a selective interference that angry voice interfered with judgment of happy face was found for the in-group judgment of Japanese participants. Also, the interference effect of to-be-ignored face on the judgment of vocal expressions was smaller in the Japanese group than in the Dutch group. The results suggest that Japanese are more tuned to vocal processing in the multisensory perception of emotion, regardless of whether the difficulty is matched between judgments of facial and vocal expressions. In Experiment 2, the effect of to-be-ignored voice was larger for the in-group than out-group speakers when judging the facial expression, whereas the opposite result was found in the vocal judgment. The results suggest that the relative weight of facial and vocal expressions can be modulated by the familiarity with the speakers.

Index Terms: emotion perception, cultural difference


  1. Tanaka, A., Koizumi, A., Imai, H., Hiramatsu, S., Hiramoto, E., de Gelder, B. "I feel your voice: Cultural differences in the multisensory perception of emotion,” Psychological Science, in press.

Full Paper

Bibliographic reference.  Tanaka, Akihiro / Koizumi, Ai / Imai, Hisato / Hiramatsu, Saori / Hiramoto, Eriko / Gelder, Beatrice de (2010): "Cross-cultural differences in the multisensory perception of emotion", In AVSP-2010, paper S3-3.