In natural languages, the words within an utterance are often correlated over large distances. Long-spanning contextual effects of this type cannot be efficiently and robustly captured by the traditional JV-gram approaches of stochastic language modelling. We present a new kind of stochastic grammar - the permugram model. A permugram model is obtained by linear interpolation of a large number of conventional bigram, trigram, or polygram models which operate on different permutations of the input word sequence under consideration. This way, stochastic dependences between word pairs or word triples lying adjacent as well as remote in the input text can be captured simultaneously without the requirement of very large iNT-grams. Using the permugram model, we achieved test set perplexity reductions of 5-10% compared with interpolated JV-gram models, depending on the application.
Bibliographic reference. Schukat-Talamazzini, Ernst-Günter / Hendrych, R. / Kompe, Ralf / Niemann, Heinrich (1995): "Permugram language models", In EUROSPEECH-1995, 1773-1776.