FAAVSP - The 1st Joint Conference on Facial Analysis, Animation, and
Auditory-Visual Speech Processing

Vienna, Austria
September 11-13, 2015

Optimal Timing of Audio-Visual Text Presentation: The Role of Attention

Maiko Takahashi (1,2), Akihiro Tanaka (2)

(1) Japan Society for the Promotion of Science; (2) Tokyo Woman’s Christian University; Tokyo, Japan

This study investigates the optimal timing of audio-visual presentation for text comprehension. #In Experiment 1, participants were asked to read and/or hear the expository passages in three conditions; visually presented, auditorily presented, and audio-visually presented condition. Comprehension performance in audio-visual presentation condition was not different from that in visual or auditory condition, raising the possibility that cognitive load on processing both visual and auditory information negated the positive effect of multi-modal presentation. To reduce the load of processing dual information simultaneously, we proposed to delay the iming of one of the two modality presentation and to direct participants’ attention to one of the two modality information during audio-visual text presentation. In Experiment 2 to 4, passages were presented audio-visually in three conditions; auditorily preceding, simultaneous, and visually preceding conditions. Participants were instructed to direct their attention to whole information (Exp. 2), visual information (Exp. 3), and auditory information (Exp. 4). The results showed that comprehension performance was higher in the visually preceding condition when their attention was directed whole or visual information. Based on the results, the integration process of audio-visually presented text information was discussed. Index Terms: reading comprehension, listening comprehension, audio-visual integration, multimedia learning

Full Paper

Bibliographic reference.  Takahashi, Maiko / Tanaka, Akihiro (2015): "Optimal timing of audio-visual text presentation: the role of attention", In FAAVSP-2015, 185-189.