Auditory-Visual Speech Processing (AVSP) 2009

University of East Anglia, Norwich, UK
September 10-13, 2009

Visual Influence on Auditory Perception: Is Speech Special?

Christian Kroos, Katherine Hogan

MARCS Auditory Laboratories, University of Western Sydney, Australia

Few studies have investigated whether visual perception can influence auditory perception outside the speech domain in a way comparable to the McGurk effect. Here we used common environmental events, a wooden and a metal spoon hitting a granite surface, to test for a change in auditory perception of the spoon’s material induced by an incongruent video. To obtain a sensitive measure and avoid directing the participants’ attention towards the stimulus manipulation, an AXB matching task was employed where in the target trials all three auditory stimuli were the same, while on the visual side the X stimulus was presented with no video and A and B were presented with a control video and a video either showing the wooden or the metal spoon. No visual influence was found. A second even more sensitive test investigated whether two neighbouring stimuli of an acoustic wood-to-metal continuum that were found to be auditorily non-discriminable became discriminable when one of them was enforced with the appropriate visual stimulus. Again, no effect was found. We interpreted the results as showing that the McGurk effect is limited to vocal-tract-like events.

Index Terms: cross-modal perception, McGurk effect, nonspeech events

Full Paper

Bibliographic reference.  Kroos, Christian / Hogan, Katherine (2009): "Visual influence on auditory perception: is speech special?", In AVSP-2009, 70-75.