EUROSPEECH 2001 Scandinavia
7th European Conference on Speech Communication and Technology

Aalborg, Denmark
September 3-7, 2001


Is This Conversation on Track?

Paul Carpenter, Chun Jin, Daniel Wilson, Rong Zhang, Dan Bohus, Alexander I. Rudnicky

Carnegie Mellon University, USA

Confidence annotation allows a spoken dialog system to accurately assess the likelihood of misunderstanding at the utterance level and to avoid breakdowns in interaction. We describe experiments that assess the utility of features from the decoder, parser and dialog levels of processing. We also investigate the effectiveness of various classifiers, including Bayesian Networks, Neural Networks, SVMs, Decision Trees, AdaBoost and Naive Bayes, to combine this information into an utterance-level confidence metric. We found that a combination of a subset of the features considered produced promising results with several of the classification algorithms considered, e.g., our Bayesian Network classifier produced a 45.7% relative reduction in confidence assessment error and a 29.6% reduction relative to a handcrafted rule.

Full Paper

Bibliographic reference.  Carpenter, Paul / Jin, Chun / Wilson, Daniel / Zhang, Rong / Bohus, Dan / Rudnicky, Alexander I. (2001): "Is this conversation on track?", In EUROSPEECH-2001, 2121.