FAAVSP - The 1st Joint Conference on Facial Analysis, Animation, and
Auditory-Visual Speech Processing

Vienna, Austria
September 11-13, 2015

Audiovisual Binding in Speech Perception

Jean-Luc Schwartz

GIPSA Lab, Grenoble, France

We have been elaborating in the last years in Grenoble a series of experimental works in which we attempt to show that audiovisual speech perception comprises an "audiovisual binding" stage before fusion and decision. This stage would be in charge to extract and associate the auditory and visual cues corresponding to a given speech source, before further categorisation processes could take place at a higher stage. We developed paradigms to characterize audiovisual binding in terms of both "streaming" and "chunking" adequate pieces of information. This can lead to elements of a possible computational model, in relation with a larger theoretical perceptuo-motor framework for speech perception, the "Perception-for-Action-Control" Theory.

Jean-Luc Schwartz, Research Director at CNRS, has been leading ICP (Institut de la Communication Parlée, Grenoble France) from 2003 to 2006. His main areas of research involve perceptual processing, perceptuo-motor interactions, audiovisual speech perception, phonetic bases of phonological systems and the emergence of language, with publications in cognitive psychology (e.g. Cognition, Perception & Psychophysics, Behavioral & Brain Sciences, Hearing Research), neurosciences (e.g. Neuroimage or Human Brain Mapping), signal processing and computational modelling (e.g. IEEE Trans. Speech and Audio Processing, JASA, C omputer Speech and Language, Language and Cognitive Processes), and phonetics in relation with phonology (e.g. Journal of Phonetics or Phonology Laboratory). He has been involved in many national and European projects, and responsible of some of them. He coordinated a number of special issues of journals such as Speech Communication, Primatology, Philosophical Transactions of the Royal Society B, Frontiers in Psychology, Journal of Phonetics. He organized several international workshops on Audiovisual Speech Processing, Language Emergence or Face-to-Face Communication.

Bibliographic reference.  Schwartz, Jean-Luc (2015): "Audiovisual binding in speech perception", In FAAVSP-2015 (abstract).