Ninth International Conference on Spoken Language Processing

Pittsburgh, PA, USA
September 17-21, 2006

A Spanish Speech to Sign Language Translation System for Assisting Deaf-Mute People

R. San-Segundo, R. Barra, L. F. D’Haro, J. M. Montero, R. Córdoba, J. Ferreiros

Universidad Politécnica de Madrid, Spain

This paper describes the first experiments of a speech to sign language translation system in a real domain. The developed system is focused on the sentences spoken by an officer when assisting people in applying for, or renewing the National Identification Document (NID) and the Passport. This system translates officer explanations into sign language for deaf-mute people. The translation system is composed by a speech recognizer (for decoding the spoken utterance into a word sequence), a natural language translator (for converting a word sequence into a sequence of gestures belonging to the sign language), and a 3D avatar animation module (for playing the gestures). The field experiments have reported a 27.2% GER (Gesture Error Rate) and a 0.62 BLEU (BiLingual Evaluation Understudy).

Full Paper

Bibliographic reference.  San-Segundo, R. / Barra, R. / D’Haro, L. F. / Montero, J. M. / Córdoba, R. / Ferreiros, J. (2006): "A Spanish speech to sign language translation system for assisting deaf-mute people", In INTERSPEECH-2006, paper 1243-Tue3CaP.12.