EUROSPEECH '95

Currently the errorrate is the most widely used general measure of performance for identification experiments. However, the errorrate gives no information about the distribution of errors over the available response categories. Starting from the definition of perplexity of the observations in a confusion matrix, a new measure is introduced, the errordispersion, that is normalized with respect to errorrate. The errordispersion can be interpreted as the effective number of error categories per stimulus or response. Furthermore, a technique is introduced to estimate the difference between confusion matrices as a fraction of unique observations or errors. With examples from the literature, it is shown that errordispersion can point out similarities in cases where errorrate varies, and can point out differences when errorrates are similar.
Bibliographic reference. Son, Rob J. J. H. van (1995): "A method to quantify the error distribution in confusion matrices", In EUROSPEECH1995, 22772280.