Punctuation Prediction in Spontaneous Conversations: Can We Mitigate ASR Errors with Retrofitted Word Embeddings?

Łukasz Augustyniak, Piotr Szymański, Mikołaj Morzy, Piotr Żelasko, Adrian Szymczak, Jan Mizgajski, Yishay Carmiel, Najim Dehak


Automatic Speech Recognition (ASR) systems introduce word errors, which often confuse punctuation prediction models, turning punctuation restoration into a challenging task. These errors usually take the form of homophones (words which share exact or almost exact pronunciation but differ in meaning) and oronyms (homophones which consist of multiple words). We show how retrofitting of the word embeddings on the domain-specific data can mitigate ASR errors. Our main contribution is a method for a better alignment of homophone embeddings and the validation of the presented method on the punctuation prediction task. We record the absolute improvement in punctuation prediction accuracy between 6.2% (for question marks) to 9% (for periods) when compared with the state-of-the-art model.


 DOI: 10.21437/Interspeech.2020-1250

Cite as: Augustyniak, Ł., Szymański, P., Morzy, M., Żelasko, P., Szymczak, A., Mizgajski, J., Carmiel, Y., Dehak, N. (2020) Punctuation Prediction in Spontaneous Conversations: Can We Mitigate ASR Errors with Retrofitted Word Embeddings?. Proc. Interspeech 2020, 4906-4910, DOI: 10.21437/Interspeech.2020-1250.


@inproceedings{Augustyniak2020,
  author={Łukasz Augustyniak and Piotr Szymański and Mikołaj Morzy and Piotr Żelasko and Adrian Szymczak and Jan Mizgajski and Yishay Carmiel and Najim Dehak},
  title={{Punctuation Prediction in Spontaneous Conversations: Can We Mitigate ASR Errors with Retrofitted Word Embeddings?}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={4906--4910},
  doi={10.21437/Interspeech.2020-1250},
  url={http://dx.doi.org/10.21437/Interspeech.2020-1250}
}