4th International Conference on Spoken Language Processing

Philadelphia, PA, USA
October 3-6, 1996

RWC Multimodal Database for Interactions by Integration of Spoken Language and Visual Information

Satoru Hayamizu, Osamu Hasegawa, Katunobu Itou, Katuhiko Sakaue, Kazuyo Tanaka, Shigeki Nagaya, Masayuki Nakazawa, T. Endoh, Fumio Togawa, Kenji Sakamoto, Kazuhiko Yamamoto

Electrotechnical Laboratory, Tsukuba, Japan

This paper describes our design policy and prototype data collection of RWC (Real World Computing Program) multimodal database. The database is intended for research and development on the integration of spoken language and visual information for human computer interactions. The interactions are supposed to use image recognition, image synthesis, speech recognition, and speech synthesis. Visual information also includes non-verbal communication such as interactions using hand gestures and facial expressions be- tween human and a human-like CG (Computer Graphics) agent with a face and hands. Based on the experiments of interactions with these modes, specifications of the database are discussed from the viewpoint of controlling the variability and cost for the collection.

Full Paper

Bibliographic reference.  Hayamizu, Satoru / Hasegawa, Osamu / Itou, Katunobu / Sakaue, Katuhiko / Tanaka, Kazuyo / Nagaya, Shigeki / Nakazawa, Masayuki / Endoh, T. / Togawa, Fumio / Sakamoto, Kenji / Yamamoto, Kazuhiko (1996): "RWC multimodal database for interactions by integration of spoken language and visual information", In ICSLP-1996, 2171-2174.