6th SIGdial Workshop on Discourse and Dialogue

Lisbon, Portugal
September 2-3, 2005

Partially Observable Markov Decision Processes with Continuous Observations for Dialogue Management

Jason D. Williams (1), Pascal Poupart (2), Steve Young (1)

(1) Cambridge University, Engineering Department, Cambridge, UK
(2) School of Computer Science, University of Waterloo, Canada

This work shows how a dialogue model can be represented as a Partially Observable Markov Decision Process (POMDP) with observations composed of a discrete and continuous component. The continuous component enables the model to directly incorporate a confidence score for automated planning. Using a testbed simulated dialogue management problem, we show how recent optimization techniques are able to find a policy for this continuous POMDP which outperforms a traditional MDP approach. Further, we present a method for automatically improving handcrafted dialogue managers by incorporating POMDP belief state monitoring, including confidence score information. Experiments on the testbed system show significant improvements for several example handcrafted dialogue managers across a range of operating conditions.

Full Paper

Bibliographic reference.  Williams, Jason D. / Poupart, Pascal / Young, Steve (2005): "Partially observable Markov decision processes with continuous observations for dialogue management", In SIGdial6-2005, 25-34.