Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition

Chao Weng, Chengzhu Yu, Jia Cui, Chunlei Zhang, Dong Yu


In this work, we propose minimum Bayes risk (MBR) training of RNN-Transducer (RNN-T) for end-to-end speech recognition. Specifically, initialized with a RNN-T trained model, MBR training is conducted via minimizing the expected edit distance between the reference label sequence and on-the-fly generated N-best hypothesis. We also introduce a heuristic to incorporate an external neural network language model (NNLM) in RNN-T beam search decoding and explore MBR training with the external NNLM. Experimental results demonstrate an MBR trained model outperforms a RNN-T trained model substantially and further improvements can be achieved if trained with an external NNLM. Our best MBR trained system achieves absolute character error rate (CER) reductions of 1.2% and 0.5% on read and spontaneous Mandarin speech respectively over a strong convolution and transformer based RNN-T baseline trained on ~21,000 hours of speech.


 DOI: 10.21437/Interspeech.2020-1221

Cite as: Weng, C., Yu, C., Cui, J., Zhang, C., Yu, D. (2020) Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition. Proc. Interspeech 2020, 966-970, DOI: 10.21437/Interspeech.2020-1221.


@inproceedings{Weng2020,
  author={Chao Weng and Chengzhu Yu and Jia Cui and Chunlei Zhang and Dong Yu},
  title={{Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={966--970},
  doi={10.21437/Interspeech.2020-1221},
  url={http://dx.doi.org/10.21437/Interspeech.2020-1221}
}