Third International Conference on Spoken Language Processing (ICSLP 94)

Yokohama, Japan
September 18-22, 1994

Labeler Agreement in Phonetic Labeling of Continuous Speech

Ronald Cole, Beatrice T. Oshika, Mike Noel, Terri Lander, Mark Fanty

Center for Spoken Language Understanding, Oregon Graduate Institute of Science & Technology, Portland, OR, USA

This paper analyzes inter-labeler agreement of label choice and boundary placement for human phonetic transcriptions of continuous telephone speech in different languages. In experiment one, English, German, Mandarin and Spanish are labeled by fluent speakers of the languages. In experiment two, German and Hindi are labeled by linguists who do not speak the languages. Experiment two uses a somewhat finer phonetic transcription set than experiment one. We compare the transcriptions of the utterances in terms of the minimum number of substitutions, insertions and deletions needed to map one transcription to the other. Native speakers agree on the average 67.52% of the time at the finest level of labeling, including diacritics. Non-native linguists agree 34.41% of the time. The implications of the results are discussed for evaluation of phonetic recognition algorithms.

Full Paper

Bibliographic reference.  Cole, Ronald / Oshika, Beatrice T. / Noel, Mike / Lander, Terri / Fanty, Mark (1994): "Labeler agreement in phonetic labeling of continuous speech", In ICSLP-1994, 2131-2134.