Acoustic cue variability affects eye movement behaviour during non-native speech perception: a GAMM model

Jessie S. Nixon, Catherine T. Best


Participants in the ‘visual world’ paradigm simultaneously process both auditory and visual cues in order to match speech to target images. Previous research has shown that when native speakers listen to speech that has high within-category variability in the contrastive dimension, auditory perceptual uncertainty increases, resulting in increased looks to competitor objects. This suggests a cross-modal effect, where reduced reliability in the auditory domain leads to increased search for evidence in the visual domain. The present study investigated the effects of within- category acoustic variability on eye movements during the acquisition of a new acoustic dimension not present in the native language, namely English speakers’ acquisition of lexical tone. All participants heard a bimodal distribution of stimuli, with distribution peaks at the prototypical pitch values for high and mid tones; however, presentation frequency differed between conditions: high-variance vs. low-variance. Based on previous research, we expected lower uncertainty and better learning in the low-variance condition. GAMM models of eye movement data showed that within- category acoustic variance increases perceptual uncertainty in the auditory domain and hinders acquisition of a cue dimension. The results also show a cross-modal effect: lower relibility in the auditory domain leads to increased search for cues in the visual domain, even when visual cues are held constant across conditions.


 DOI: 10.21437/AVSP.2017-2

Cite as: Nixon, J.S., Best, C.T. (2017) Acoustic cue variability affects eye movement behaviour during non-native speech perception: a GAMM model. Proc. The 14th International Conference on Auditory-Visual Speech Processing, 6-11, DOI: 10.21437/AVSP.2017-2.


@inproceedings{Nixon2017,
  author={Jessie S. Nixon and Catherine T. Best},
  title={ Acoustic cue variability affects eye movement behaviour during non-native speech perception: a GAMM model},
  year=2017,
  booktitle={Proc. The 14th International Conference on Auditory-Visual Speech Processing},
  pages={6--11},
  doi={10.21437/AVSP.2017-2},
  url={http://dx.doi.org/10.21437/AVSP.2017-2}
}