5th International Conference on Spoken Language Processing

Sydney, Australia
November 30 - December 4, 1998

Estimating Entropy of a Language from Optimal Word Insertion Penalty

Kazuya Takeda, Atsunori Ogawa, Fumitada Itakura

Nagoya University, Japan

The relationship between the optimal value of word insertion penalty and entropy of the language is discussed, based on the hypothesis that the optimal word insertion penalty compensates the probability given by a language model to the true probability. It is shown that the optimal word insertion penalty can be calculated as the difference between test set entropy of the given language model and true entropy of the given test set sentences. The correctness of the idea is confirmed through recognition experiment, where the entropy of the given set of sentences are estimated from two different language models and word insertion penalty optimized for each language model.

Full Paper

Bibliographic reference.  Takeda, Kazuya / Ogawa, Atsunori / Itakura, Fumitada (1998): "Estimating entropy of a language from optimal word insertion penalty", In ICSLP-1998, paper 0456.