5th International Conference on Spoken Language Processing
Combining different language models is an important task. Linear interpolation is the established method to do this. We will present a new method called log-linear interpolation (LLI) which combines the simplicity of linear interpolation with essential parts from maximum-entropy models by linearly interpolating the scores of different models. The first series of experiments focuses on adaptation. Unigram, bigram and trigram models trained on NAB are combined with unigram and bigram models trained on a small domain specific corpus. LLI compares favorably with linear interpolation. The second series combines bigram and distance-bigram models. Here, relative improvements are larger (~20% in perplexity). This task seems to be the ideal application of LLI. To further scrutinize the method, first frequent pairs of words are joined to phrases and next, bigram and distance-bigram are combined by LLI. This experiment yields perplexities just 2.5% above the original trigram perplexity.
Bibliographic reference. Klakow, Dietrich (1998): "Log-linear interpolation of language models", In ICSLP-1998, paper 0522.