5th International Conference on Spoken Language Processing
Manual Cued Speech (MCS) is an effective method of communication by the deaf and hearing-impaired. We describe our work on assessing the feasibility of automatic determination and presentation of cues without intervention by the speaker. The conclusions of this study are applied to the design and implementation of a prototype automatic cueing system using HMM-based automatic speech recognition software to identify the cues in real time. We also describe the features of our cue display that enhance its effectiveness such as style of cue images and the timing of their transitions. Our experiments show keyword reception by experienced MCS users improves significantly with the use of our system (66%) relative to speechreading alone (35%) on low-context sentences.
Video #1 [.mpg] - The manually cued sentence "The old castle passed from the duke to the king."
Video #2 [.mpg] - Automatically cued (discrete cues) sentence "The loss and two wins were fair games."
Video #3 [.mpg] - Automatically cued (dynamic cues) sentence "The kite may fly on this windy day."
Bibliographic reference. Duchnowski, Paul / Braida, Louis / Bratakos, Maroula / Lum, David / Sexton, Matthew / Krause, Jean (1998): "A speechreading aid based on phonetic ASR", In ICSLP-1998, paper 0589.