5th International Conference on Spoken Language Processing

Sydney, Australia
November 30 - December 4, 1998

Soft State-Tying for HMM-based Speech Recognition

Christoph Neukirchen, Daniel Willett, Gerhard Rigoll

Department of Computer Science, Gerhard-Mercator-University Duisburg, Germany

This paper introduces a method for regularization of HMM systems that avoids parameter overfitting caused by insufficient training data. Regularization is done by augmenting the EM training method by a penalty term that favors simple and smooth HMM systems. The penalty term is constructed as a mixture model of negative exponential distributions that is assumed to generate the state dependent emission probabilities of the HMMs. This new method is the successful transfer of a well known regularization approach in neural networks to the HMM domain and can be interpreted as a generalization of traditional state-tying for HMM systems. The effect of regularization is demonstrated for continuous speech recognition tasks by improving overfitted triphone models and by speaker adaptation with limited training data.

Full Paper

Bibliographic reference.  Neukirchen, Christoph / Willett, Daniel / Rigoll, Gerhard (1998): "Soft state-tying for HMM-based speech recognition", In ICSLP-1998, paper 0346.