Ninth International Conference on Spoken Language Processing

Pittsburgh, PA, USA
September 17-21, 2006

Measuring the Acceptable Word Error Rate of Machine-Generated Webcast Transcripts

Cosmin Munteanu (1), Gerald Penn (1), Ron Baecker (1), Elaine Toms (2), David James (1)

(1) University of Toronto, Canada; (2) Dalhousie University, Canada

The increased availability of broadband connections has recently led to an increase in the use of Internet broadcasting (webcasting). Most webcasts are archived and accessed numerous times retrospectively. One of the hurdles users face when browsing and skimming through archives is the lack of text transcripts of the audio channel of the webcast archive. In this paper, we proposed a procedure for prototyping an Automatic Speech Recognition (ASR) system that generates realistic transcripts of any desired Word Error Rate (WER), thus overcoming the drawbacks of both prototype-based and Wizard of Oz simulations. We used such a system in a study where human subjects perform question-answering tasks using archives of webcast lectures, and showed that their performance and perception of transcript quality is linearly affected by WER, and that transcripts of WER equal or less than 25% would be acceptable for use in webcast archives.

Full Paper

Bibliographic reference.  Munteanu, Cosmin / Penn, Gerald / Baecker, Ron / Toms, Elaine / James, David (2006): "Measuring the acceptable word error rate of machine-generated webcast transcripts", In INTERSPEECH-2006, paper 1756-Mon1CaP.2.