4th International Conference on Spoken Language Processing

Philadelphia, PA, USA
October 3-6, 1996

Lipreading Supplemented by Voice Fundamental Frequency: To What Extent Does the Addition of Voicing Increase Lexical Uniqueness for the Lipreader?

Edward T. Auer Jr., Lynne E. Bernstein

Spoken Language Processes Laboratory, House Ear Institute, Los Angeles, CA, USA

Lipreading in combination with an acoustic indication of voice fundamental frequency (F0) has been shown to greatly enhance word recognition accuracy with sentence stimuli [1]. A possible explanation for this effect is that F0 delivers information for consonantal voicing. In Experiment 1, we showed with a computational model how voicing information affects the uniqueness of lipread words in a large phonemically transcribed machine-readable lexicon. In Experiment 2, the same computational methods were used to simulate the results obtained by McGrath and Summerfield [2] for lipreading with and without acoustic F0. The model failed to account in full for the behaviorally observed enhancements. It is suggested that lexical biasing in word recognition can account for the difference between the model and the behavioral results. (This work was supported by NIH Grant DC-00695.)

Full Paper

Bibliographic reference.  Auer Jr., Edward T. / Bernstein, Lynne E. (1996): "Lipreading supplemented by voice fundamental frequency: to what extent does the addition of voicing increase lexical uniqueness for the lipreader?", In ICSLP-1996, 86-89.