13th Annual Conference of the International Speech Communication Association

Portland, OR, USA
September 9-13, 2012

Real-Time Lecture Transcription using ASR for Czech Hearing Impaired or Deaf Students

Petr Cerva, Jan Silovsky, Jindrich Zdansky, Jan Nouza, Jiri Malek

Institute of Information Technology and Electronics, Faculty of Mechatronics, Technical University of Liberec, Liberec, Czech Republic

This paper describes a client-server system developed to enable hearing impaired persons to participate in lectures by providing real-time displayed transcripts. The core of this system is formed by an ASR module running on a recognition server and processing the input audio-video stream. This engine utilizes a large lexicon, topic-specific language models mixed properly from various sources (e.g. transcripts of spontaneous utterances, theses, web discussions) and unsupervised incremental speaker adaptation methods to cope with spontaneous lecture speech in highly inflective Czech language. The raw output of the ASR module is converted into a more readable form using a developed post-processing module based on finite state transducers. The resulting formated text (i.e. containing punctuation marks, digit strings, etc) is then displayed on the screen of each client device (e.g. notebook or tablet) in the lecture room.

Index Terms: real-time lecture transcription, applications for handicapped persons, applications in learning

Full Paper

Bibliographic reference.  Cerva, Petr / Silovsky, Jan / Zdansky, Jindrich / Nouza, Jan / Malek, Jiri (2012): "Real-time lecture transcription using ASR for Czech hearing impaired or deaf students", In INTERSPEECH-2012, 763-766.