EUROSPEECH 2001 Scandinavia
7th European Conference on Speech Communication and Technology

Aalborg, Denmark
September 3-7, 2001


Word Level Confidence Annotation using Combinations of Features

Rong Zhang, Alexander I. Rudnicky

Carnegie Mellon University, USA

This paper describes the development of a word-level confidence metric suitable for use in a dialog system. Two aspects of the problems are investigated: the identification of useful features and the selection of an effective classifier. We find that two parse-level features, Parsing-Mode and Slot-Backoff-Mode, provide annotation accuracy comparable to that observed for decoder-level features. However, both decoder-level and parse-level features independently contribute to confidence annotation accuracy. In comparing different classification techniques, we found that Support Vector Machines (SVMs) appear to provide the best accuracy. Overall we achieve 39.7% reduction in annotation uncertainty for a binary confidence decision in a travel-planning domain.

Full Paper

Bibliographic reference.  Zhang, Rong / Rudnicky, Alexander I. (2001): "Word level confidence annotation using combinations of features", In EUROSPEECH-2001, 2105-2108.