EUROSPEECH 2001 Scandinavia
7th European Conference on Speech Communication and Technology

Aalborg, Denmark
September 3-7, 2001


Separating Three Simultaneous Speeches with Two Microphones by Integrating Auditory and Visual Processing

Hiroshi G. Okuno, Kazuhiro Nakadai, Tino Lourens, Hiroaki Kitano

Kitano Symbiotic Systems Project, JST, Japan

This paper addresses the problem of automatic recognition of three simultaneous speeches with two microphones, that is, that of sound source separation where the number of sound sources is greater than that of microphones. The approach used is the {it direction-pass filter}, which is implemented by hypothetical reasoning on the interaural phase difference (IPD) and interaural intensity difference (IID). Auditory processing calculates IPD and IID for each subband, and generates hypotheses for precalculated IPD and IID for every direction including one obtained by visual processing. Then the system calculates the belief factor of hypothesis by Dempster-Shafer theory and determines the direction of each subband. Subbands of the specific direction are collected and then converted to a wave form by inverse FFT. With 200 benchmarks of three simultaneous utterances of Japanese words, the average 1-best and 10-best recognition rates of extracted speeches are 60% and 81%, respectively.

Full Paper

Bibliographic reference.  Okuno, Hiroshi G. / Nakadai, Kazuhiro / Lourens, Tino / Kitano, Hiroaki (2001): "Separating three simultaneous speeches with two microphones by integrating auditory and visual processing", In EUROSPEECH-2001, 2643-2646.