5th European Conference on Speech Communication and Technology

Rhodes, Greece
September 22-25, 1997

Spoken Dialogue System Evaluation: A First Framework for Reporting Results

Norman M. Fraser

Department of Linguistic and International Studies University of Surrey, Surrey, United Kingdom

There are no agreed standards for reporting the performance of spoken dialogue systems. This paper proposes a core set of metrics to be used for this purpose. For this set, operational definitions are supplied, to regularise their application. The intention in proposing this framework is not that it should be exhaustive, nor that it should be perfect, but rather that it should provide a practical starting point, thereby allowing initial system comparison to be achieved quickly and with some measure of confidence.

Full Paper

Bibliographic reference.  Fraser, Norman M. (1997): "Spoken dialogue system evaluation: a first framework for reporting results", In EUROSPEECH-1997, 1907-1910.