Ninth International Conference on Spoken Language Processing

Pittsburgh, PA, USA
September 17-21, 2006

The ICSI+ Multilingual Sentence Segmentation System

M. Zimmerman (1), Dilek Hakkani-Tür (1), J. Fung (1), N. Mirghafori (1), L. Gottlieb (1), Elizabeth Shriberg (2), Yang Liu (3)

(1) International Computer Science Institute, USA; (2) SRI International, USA; (3) University of Texas at Dallas, USA

The ICSI+ multilingual sentence segmentation with results for English and Mandarin broadcast news automatic speech recognizer transcriptions represents a joint effort involving ICSI, SRI, and UT Dallas. Our approach is based on using hidden event language models for exploiting lexical information, and maximum entropy and boosting classifiers for exploiting lexical, as well as prosodic, speaker change and syntactic information. We demonstrate that the proposed methodology including pitch- and energy-related prosodic features performs significantly better than a baseline system that uses words and simple pause features only. Furthermore, the obtained improvements are consistent across both languages, and no language-specific adaptation of the methodology is necessary. The best results were achieved by combining hidden event language models with a boosting-based classifier that to our knowledge has not previously been applied for this task.

Full Paper

Bibliographic reference.  Zimmerman, M. / Hakkani-Tür, Dilek / Fung, J. / Mirghafori, N. / Gottlieb, L. / Shriberg, Elizabeth / Liu, Yang (2006): "The ICSI+ multilingual sentence segmentation system", In INTERSPEECH-2006, paper 1808-Mon1BuP.4.