Emojive! Collecting Emotion Data from Speech and Facial Expression Using Mobile Game App

Ji Ho Park, Nayeon Lee, Dario Bertero, Anik Dey, Pascale Fung


We developed Emojive!, a mobile game app to make emotion recognition from audio and image interactive and fun, motivating the users to play with the app. The game is to act out a specific emotion, among six emotion labels (happy, sad, anger, anxiety, loneliness, criticism), given by the system. Double player mode lets two people to compete their acting skills. The more users play the game, the more emotion-labelled data will be acquired. We are using deep Convolutional Neural Network (CNN) models to recognize emotion from audio and facial image in real-time with a mobile front-end client including intuitive user interface and simple data visualization.


Cite as: Park, J.H., Lee, N., Bertero, D., Dey, A., Fung, P. (2017) Emojive! Collecting Emotion Data from Speech and Facial Expression Using Mobile Game App. Proc. Interspeech 2017, 827-828.


@inproceedings{Park2017,
  author={Ji Ho Park and Nayeon Lee and Dario Bertero and Anik Dey and Pascale Fung},
  title={Emojive! Collecting Emotion Data from Speech and Facial Expression Using Mobile Game App},
  year=2017,
  booktitle={Proc. Interspeech 2017},
  pages={827--828}
}