Second European Conference on Speech Communication and Technology

Genova, Italy
September 24-26, 1991


Outline of a Design-Oriented Evaluation Framework for Speech-Driven Applications

Paulus H. Vossen

Fraunhofer Institut für Arbeitswirtschaft und Organisation, Stuttgart, Baden-Wtirttemberg, Germany

The basic technologies for speech input and output are ready for large-scale implementation, though improvements in several areas will certainly be made. What is still missing is an integrated framework for evaluation and design, in which the existing know-how about speech input and output ergonomics can be embedded. In this paper we describe the components of such a framework and a strategy for design-oriented system evaluation. We will focus on formative evaluation, i. e. a comprehensive scheme of assessment which can be used during speech system design. The proposed approach has been accepted within the ESPRIT project SUNSTAR as the common framework for assessment. Keywords: speech input and output assessment, dialogue evaluation, application evaluation, evaluation strategy, evaluation layers, evaluation criteria, evaluation methods

Full Paper

Bibliographic reference.  Vossen, Paulus H. (1991): "Outline of a design-oriented evaluation framework for speech-driven applications", In EUROSPEECH-1991, 517-520.