An Investigation of Cross-Cultural Semi-Supervised Learning for Continuous Affect Recognition

Adria Mallol-Ragolta, Nicholas Cummins, Björn W. Schuller


One of the keys for supervised learning techniques to succeed resides in the access to vast amounts of labelled training data. The process of data collection, however, is expensive, time-consuming, and application dependent. In the current digital era, data can be collected continuously. This continuity renders data annotation into an endless task, which potentially, in problems such as emotion recognition, requires annotators with different cultural backgrounds. Herein, we study the impact of utilising data from different cultures in a semi-supervised learning approach to label training material for the automatic recognition of arousal and valence. Specifically, we compare the performance of culture-specific affect recognition models trained with manual or cross-cultural automatic annotations. The experiments performed in this work use the dataset released for the Cross-cultural Emotion Sub-challenge of the Audio/Visual Emotion Challenge (AVEC) 2019. The results obtained convey that the cultures used for training impact on the system performance. Furthermore, in most of the scenarios assessed, affect recognition models trained with hybrid solutions, combining manual and automatic annotations, surpass the baseline model, which was exclusively trained with manual annotations.


 DOI: 10.21437/Interspeech.2020-2641

Cite as: Mallol-Ragolta, A., Cummins, N., Schuller, B.W. (2020) An Investigation of Cross-Cultural Semi-Supervised Learning for Continuous Affect Recognition. Proc. Interspeech 2020, 511-515, DOI: 10.21437/Interspeech.2020-2641.


@inproceedings{Mallol-Ragolta2020,
  author={Adria Mallol-Ragolta and Nicholas Cummins and Björn W. Schuller},
  title={{An Investigation of Cross-Cultural Semi-Supervised Learning for Continuous Affect Recognition}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={511--515},
  doi={10.21437/Interspeech.2020-2641},
  url={http://dx.doi.org/10.21437/Interspeech.2020-2641}
}