Detection of Voicing and Place of Articulation of Fricatives with Deep Learning in a Virtual Speech and Language Therapy Tutor

Ivo Anjos, Maxine Eskenazi, Nuno Marques, Margarida Grilo, Isabel Guimarães, João Magalhães, Sofia Cavaco


Children with fricative distortion errors have to learn how to correctly use the vocal folds, and which place of articulation to use in order to correctly produce the different fricatives. Here we propose a virtual tutor for fricatives distortion correction. This is a virtual tutor for speech and language therapy that helps children understand their fricative production errors and how to correctly use their speech organs. The virtual tutor uses log Mel filter banks and deep learning techniques with spectral-temporal convolutions of the data to classify the fricatives in children’s speech by place of articulation and voicing. It achieves an accuracy of 90.40% for place of articulation and 90.93% for voicing with children’s speech. Furthermore, this paper discusses a multidimensional advanced data analysis of the first layer convolutional kernel filters that validates the usefulness of performing the convolution on the log Mel filter bank.


 DOI: 10.21437/Interspeech.2020-2821

Cite as: Anjos, I., Eskenazi, M., Marques, N., Grilo, M., Guimarães, I., Magalhães, J., Cavaco, S. (2020) Detection of Voicing and Place of Articulation of Fricatives with Deep Learning in a Virtual Speech and Language Therapy Tutor. Proc. Interspeech 2020, 3156-3160, DOI: 10.21437/Interspeech.2020-2821.


@inproceedings{Anjos2020,
  author={Ivo Anjos and Maxine Eskenazi and Nuno Marques and Margarida Grilo and Isabel Guimarães and João Magalhães and Sofia Cavaco},
  title={{Detection of Voicing and Place of Articulation of Fricatives with Deep Learning in a Virtual Speech and Language Therapy Tutor}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={3156--3160},
  doi={10.21437/Interspeech.2020-2821},
  url={http://dx.doi.org/10.21437/Interspeech.2020-2821}
}