International Conference on Auditory-Visual Speech Processing 2008

Tangalooma Wild Dolphin Resort, Moreton Island, Queensland, Australia
September 26-29, 2008

Multimodal Perception of Anticipatory Behavior - Comparing Blind, Hearing and Cued Speech Subjects

Emilie Troille (1,2), Marie-Agnès Cathiard (1), Christian Abry (1), Lucie Ménard (3), Denis Beautemps (2)

(1) CRI, EA 610; (2) GIPSA-Lab-ICP, UMR 5216, CNRS-INPG-Université Stendhal, Grenoble, France
(3) Laboratoire de Phonétique, Université du Québec à Montréal, Canada

In a previous study [1] on vowel rounding anticipatory perception through a fricative consonant stream with a gating paradigm, we discovered that auditory information could be, in this VCV frame, ahead of visual information, and even of audiovisual information (hence A>AV>V). We propose here to extend this experiment to different modality-impaired groups of adults and children, namely: (i) to blind and impaired-hearing adults vs. normal hearing and sighted ones; (ii) and to impaired-hearing vs. normal hearing children (7-11- year-old). Blind and sighted adults had globally comparable results in the auditory modality, though individually less categorical for the blind. In the visual condition, hearingimpaired adults had the same delayed performance as normal. In addition the same hearing-impaired adults were tested when practicing French Cued Speech (CS). Their performance indicated clearly that, with CS, they succeeded in the identification task as precociously as normal hearing in the auditory condition. Normal hearing children obtained the same ranking in their perceptual modalities, i.e. A>AV>V, but with a global delay of 20-30 ms compared to adults. Hearingimpaired children obtained again an anticipatory gain with CS comparable to the audio of normal children. The conclusion is that each group takes the best advantage of the natural timing of their available sensory modalities, by the face, the hand and the ear. For Cued Speech coders, the hand-to-face coordination can clearly substitute in time to the ear.


  1. E. Troille, M.-A. Cathiard & C. Abry. Consequences on bimodal perception of the timing of the consonant and vowel audiovisual flows. Proceedings of AVSP 2007, 31 August - 3 September, Kasteel Groenendael, Hilvarenbeek, Pays-Bas, 281-286, 2007 (ISCA Archive,

Full Paper

Bibliographic reference.  Troille, Emilie / Cathiard, Marie-Agnès / Abry, Christian / Ménard, Lucie / Beautemps, Denis (2008): "Multimodal perception of anticipatory behavior - Comparing blind, hearing and cued speech subjects", In AVSP-2008, 63-68.