Second European Conference on Speech Communication and Technology

Genova, Italy
September 24-26, 1991


References in a Multimodal Dialogue: Towards a Unified Processing

B. Gaiffe, L. Romary, Jean-Marie Pierrel

CRIN-CNRS & INRIA, LORRAINE, Vandceuvre-les-Nancy, France

In this paper we tackle the problem of understanding references expressed through several different modes in a multimodal. dialogue system. We focus our attention on the two Natural Language and designation (by means of a mouse), for which we show that there exists two main strands of answers to this problem: the referential and the syntactical strand. We illustrate the first one from our experience within the Multiworks project and we show that both present some major drawbacks that they alone cannot solve. Moreover, since multimodal systems are usually based on NL, we show that the designation mode can also present some interesting properties which give it an important power of expression. This leads us to the conclusion that a multimodal dialogue system should be based on a unified architecture hinging on a multifold focus mechanism. Keywords: Multimodal dialogue, Natural Language, Focus, Reference.

Full Paper

Bibliographic reference.  Gaiffe, B. / Romary, L. / Pierrel, Jean-Marie (1991): "References in a multimodal dialogue: towards a unified processing", In EUROSPEECH-1991, 1481-1485.