4th International Conference on Spoken Language Processing

Philadelphia, PA, USA
October 3-6, 1996

An Experimental Japanese/English Interpreting Video Phone System

M. Karaorman (1), T. H. Applebaum (1), T. Itoh (2), M. Endo (2), Y. Ohno (2), M. Hoshimi (2), T. Kamai (3), K. Matsui (3), K. Hata (1), S. Pearson (1), Jean-Claude Junqua (1)

(1) Speech Technology Laboratory, Panasonic Technologies, Inc., Santa Barbara, CA, USA
(2) Matsushita Research Institute Tokyo, Inc.
(3) Central Research Laboratories, Matsushita Electric Industrial Co. Ltd.

In this paper we report on the architectural design issues and experiences gained while building and demonstrating an experimental interpreting video phone (IVP) system. The IVP system has been demonstrated in an internet home shopping simulation simultaneously before live audiences in Japan and the U.S. An American shop assistant and a Japanese customer engaged in task-directed dialogues, using their native languages. In addition to their direct audio/visual contact by ISDN video phone, each participant heard a translation of the remote speakerís utterances in a synthetic voice in real-time. Each site used a medium-size vocabulary, a continuous speech recognition system and a text-to-speech synthesis (TTS) system for the local language. Recognition results were transmitted over the internet to the remote site, where the corresponding translated sentence was spoken by TTS in the listenerís native language. All of the speech and language processing software components of the system were independently developed proprietary technologies of the authorsí laboratories which were integrated using commercially available hardware and communication media. Difficulties encountered in developing the system, the accommodations which were made, and other experiences gained through the process are reported in this paper.

Full Paper

Bibliographic reference.  Karaorman, M. / Applebaum, T. H. / Itoh, T. / Endo, M. / Ohno, Y. / Hoshimi, M. / Kamai, T. / Matsui, K. / Hata, K. / Pearson, S. / Junqua, Jean-Claude (1996): "An experimental Japanese/English interpreting video phone system", In ICSLP-1996, 1676-1679.