Second International Conference on Spoken Language Processing (ICSLP'92)

Banff, Alberta, Canada
October 13-16, 1992

Intelligibility of Audio-Visually Desynchronised Speech: Asymmetrical Effect of Phoneme Position

P. M. T. Smeele (1), A. C. Sittig (1), Vincent J. van Heuven (2)

(1) Dept. of Industrial Design Engineering, Delft University of Technology, Delft, The Netherlands
(2) Phonetics Laboratory, Dept. of Linguistics, Leyden University, Leyden, The Netherlands

In an earlier experiment we studied the effect of desynchronisation of visual and auditory information on the perception of nonsense CVC-words. Predictably, percent correctly recognized words was generally smaller in asynchronous conditions than in the synchronous condition. However, an asymmetrical effect was found: subjects' performance was poorer in the condition where audition preceded vision than when vision preceded audition. In order to gain insight into this asymmetrical effect and its causes, we have now examined subjects' responses to nonsense words on the phoneme level. Analysis of the individual phonemes which made up the original CVC-stimuli indicates that identification of the initial consonant was worse when vision preceded audition than vice versa. In contrast to this, identification of both the vowel and the final consonant was poorer when audition preceded vision. This effect of phoneme position may serve to explain why perception deteriorates more when audition precedes vision than when vision leads.

Full Paper

Bibliographic reference.  Smeele, P. M. T. / Sittig, A. C. / Heuven, Vincent J. van (1992): "Intelligibility of audio-visually desynchronised speech: asymmetrical effect of phoneme position", In ICSLP-1992, 65-68.