Speech Prosody 2006

Dresden, Germany
May 2-5, 2006

An fMRI Study of Multimodal Deixis: Preliminary Results on Prosodic, Syntactic, Manual and Ocular Pointing

Francesca Carota (1), Hélène Loevenbruck (1), Coriandre Vilain (1), Monica Baciu (2), Christian Abry (1), Laurent Lamalle (3), Cédric Pichat (2), Christoph Segebarth (4)

(1) Institut de la Communication Parlée, INPG, Univ. Stendhal; (2) Laboratoire de Psychologie et NeuroCognition, UPMF; (3) INSERM IFRn° 1, RMN biomedicale, CHU de Grenoble; (4) Unité Mixte INSERM / Univ. J. Fourier, U594; Grenoble, France

Deixis or pointing plays a crucial role in language acquisition and speech communication. In this paper we present an innovative fMRI approach in order to examine deixis. conceived as a unitary communicative strategy which employs different verbal and non-verbal speech devices to achieve the pragmatic goal of bringing relevant information to the interlocutors' attention. We designed a unified fMRI paradigm for multimodal deixis, integrating four conditions of verbal and non-verbal pointing: 1) prosodic focus, 2) syntactic extraction, 3) index finger pointing, 4) eye pointing. Sixteen subjects were examined while they gave oral, manual and ocular responses inside the 3T magnet imager. Preliminary results based on a random effect analysis with a group of 10 subjects show that all pointing conditions recruit a left temporo-parieto-frontal network, with respect to the control condition. Further analyses are being carried out to distinguish between different modalities of pointing.

Full Paper

Bibliographic reference.  Carota, Francesca / Loevenbruck, Hélène / Vilain, Coriandre / Baciu, Monica / Abry, Christian / Lamalle, Laurent / Pichat, Cédric / Segebarth, Christoph (2006): "An fMRI study of multimodal deixis: preliminary results on prosodic, syntactic, manual and ocular pointing", In SP-2006, paper 201.