4th International Conference on Spoken Language Processing

Philadelphia, PA, USA
October 3-6, 1996

Error Resolution During Multimodal Human-computer Interaction

Sharon Oviatt, Robert VanGent

Center for Human-Computer Communication, Department of Computer Science, Oregon Graduate Institute of Science & Technology, OR, USA

Recent research indicates clear performance advantages and a strong user preference for interacting multimodally with computers. However, in the problematic area of error resolution, possible advantages of multimodal interface design remain poorly understood. In the present research, a semi-automatic simulation method with a novel error-generation capability was used to collect within-subject data before and after recognition errors, and at different spiral depths in terms of number of repetitions required to resolve an error. Results indicated that users adopt a strategy of switching input modalities and lexical expressions when resolving errors, strategies that they use in a linguistically contrastive manner to distinguish a repetition from original failed input. Implications of these findings are discussed for the development of user-centered predictive models of linguistic adaptation during human-computer error resolution, and for the development of improved error handling in advanced recognition-based interfaces.

Full Paper

Bibliographic reference.  Oviatt, Sharon / VanGent, Robert (1996): "Error resolution during multimodal human-computer interaction", In ICSLP-1996, 204-207.