The developmental path of multisensory perception of emotion and phoneme in Japanese speakers

Hisako W. Yamamoto, Misako Kawahara, Akihiro Tanaka


The purpose of this study is to investigate the developmental path of multisensory perception of emotion and phoneme in Japanese speakers. In Experiment 1, Japanese children from age of 5 to 12 and adults were engaged in Emotion perception task, in which speakers’ emotions expressed in their face and voice were congruent or incongruent, and Phoneme perception task, in which auditory and visual phonemic information are congruent or incongruent (i.e., McGurk type movie). Children’s judgement weighting on voice information in emotion perception increased over age, whereas the weighting of audiovisual information in phoneme perception remained the same during childhood. Interestingly, adults’ emotion perception based on voice information was less than those of 11-12-year-olds. Experiment 2 examined whether adults’ multisensory perception was affected by their parenting experience. The results showed that parents who were rearing their children judged speakers’ emotion relying on voice information less than non-parents, whereas visual influence in phoneme perception was not different between parents and non- parents; adults’ multisensory perception may be affected by daily interaction with their children as for emotion judgement. Taken together, these results show differential integration processes between emotion and phoneme perception.


 DOI: 10.21437/AVSP.2017-20

Cite as: Yamamoto, H.W., Kawahara, M., Tanaka, A. (2017) The developmental path of multisensory perception of emotion and phoneme in Japanese speakers. Proc. The 14th International Conference on Auditory-Visual Speech Processing, 105-108, DOI: 10.21437/AVSP.2017-20.


@inproceedings{Yamamoto2017,
  author={Hisako W. Yamamoto and Misako Kawahara and Akihiro Tanaka},
  title={ The developmental path of multisensory perception of emotion and phoneme in Japanese speakers},
  year=2017,
  booktitle={Proc. The 14th International Conference on Auditory-Visual Speech Processing},
  pages={105--108},
  doi={10.21437/AVSP.2017-20},
  url={http://dx.doi.org/10.21437/AVSP.2017-20}
}