Second International Conference on Spoken Language Processing (ICSLP'92)

Banff, Alberta, Canada
October 13-16, 1992

Investigations into the Auditory F0 Speechreading Enhancement Effect Using a Sinusoidal Replica of the F0 Contour

Arthur Boothroyd, Robin S. Waldstein, Eddy Yeung

Graduate School, City University of New York, New York, NY, USA

Constant-amplitude sinusoidal replicas of the voice fundamental frequency (Fo) contours of video-recorded sentences were prepared off-line by a combination of automatic and manual Fo estimation. The replicas were then re-synchronized with the video portions of the original recording. In 12 normally hearing adults, the replicas were found to be as effective a supplement to speechreading as the low-pass filtered output of an electroglottograph. Both signals increased the number of words recognized in sentences of known topic by almost 40 percentage points. Using a constant-frequency version of the replicas, it was found that about 1/3 of the Fo speechreading enhancement effect could be accounted for by voicing detection alone.

Full Paper

Bibliographic reference.  Boothroyd, Arthur / Waldstein, Robin S. / Yeung, Eddy (1992): "Investigations into the auditory F0 speechreading enhancement effect using a sinusoidal replica of the F0 contour", In ICSLP-1992, 963-966.