4th International Conference on Spoken Language Processing

Philadelphia, PA, USA
October 3-6, 1996

Building Sensori-motor Prototypes from Audiovisual Exemplars

Gérard Bailly

Institut de la Communication Parlée - INPG & Université Stendhal, Grenoble, France

This paper shows how an articulatory model, able to produce acoustic signals from articulatory motion, can learn to speak, i.e. coordinate its movements in such a way that it utters meaningful sequences of sounds belonging to a given language. This complex learning procedure is accomplished in four major steps: (a) a babbling phase, where the device builds up a model of the forward transforms, i.e. the articulatory-to-audiovisual mapping; (b) an imitation stage, where it tries to reproduce a limited set of sound sequences produced by a distal "teacher"; (c) a "shaping" stage, where phonemes are associated with the most efficient sensori-motor representation; and finally, (d) a "rhythmic" phase, where it learns the appropriate coordination of the activations of these sensori-motor targets.

Full Paper
Sound Examples:   #1   #2   #3   #4  

Bibliographic reference.  Bailly, Gérard (1996): "Building sensori-motor prototypes from audiovisual exemplars", In ICSLP-1996, 957-960.