6th SIGdial Workshop on Discourse and Dialogue

Lisbon, Portugal
September 2-3, 2005

Evaluation of a Dialogue System in an Automotive Environment

Liza Hassel (1,2), Eli Hagen (2)

(1) Centre for Information and Language Processing, Ludwig Maximilian University; (2) Forschungs- und Innovationszentrum, BMW AG, Munich, Germany

In this paper we discuss features to enhance the usability of a spoken dialogue system (SDS) in an automotive environment. We describe the tests that were performed to evaluate those features, and the methods used to assess the test results. One of these methods is a modification of PARADISE, a framework for evaluating the performance of SDSs (Walker et al., 1998). We discuss its drawbacks for the evaluation of SDSs like ours, the modifications we have carried out, and the test results.

Full Paper

Bibliographic reference.  Hassel, Liza / Hagen, Eli (2005): "Evaluation of a dialogue system in an automotive environment", In SIGdial6-2005, 155-165.