13th Annual Conference of the International Speech Communication Association

Portland, OR, USA
September 9-13, 2012

Estimating Classifier Performance in Unknown Noise

Ehsan Variani (1), Hynek Hermansky (1,2)

(1) Center for Language and Speech Processing; (2) Human Language Technology Center of Excellence;
Johns Hopkins University, Baltimore, MD, USA

We propose and investigate methods for identifying regions of speech that have unexpected distortions not seen in training data. The methods do not require knowledge of correct labels and rely only on divergence between statistics of test and training data. We propose two metrics with and without probabilistic assumptions. Our experiments show that the proposed non-probabilistic method requires a relatively small amount of test data of the order of several seconds to stabilize, and correlates well with recognition error observed on the test data.

Index Terms: Unexpected distortions, confidence estimation, machine recognition of speech

Full Paper

Bibliographic reference.  Variani, Ehsan / Hermansky, Hynek (2012): "Estimating classifier performance in unknown noise", In INTERSPEECH-2012, 1800-1803.