Connecting and Comparing Language Model Interpolation Techniques

Ernest Pusateri, Christophe Van Gysel, Rami Botros, Sameer Badaskar, Mirko Hannemann, Youssef Oualil, Ilya Oparin

In this work, we uncover a theoretical connection between two language model interpolation techniques, count merging and Bayesian interpolation. We compare these techniques as well as linear interpolation in three scenarios with abundant training data per component model. Consistent with prior work, we show that both count merging and Bayesian interpolation outperform linear interpolation. We include the first (to our knowledge) published comparison of count merging and Bayesian interpolation, showing that the two techniques perform similarly. Finally, we argue that other considerations will make Bayesian interpolation the preferred approach in most circumstances.

 DOI: 10.21437/Interspeech.2019-1822

Cite as: Pusateri, E., Gysel, C.V., Botros, R., Badaskar, S., Hannemann, M., Oualil, Y., Oparin, I. (2019) Connecting and Comparing Language Model Interpolation Techniques. Proc. Interspeech 2019, 3500-3504, DOI: 10.21437/Interspeech.2019-1822.

  author={Ernest Pusateri and Christophe Van Gysel and Rami Botros and Sameer Badaskar and Mirko Hannemann and Youssef Oualil and Ilya Oparin},
  title={{Connecting and Comparing Language Model Interpolation Techniques}},
  booktitle={Proc. Interspeech 2019},