14thAnnual Conference of the International Speech Communication Association

Lyon, France
August 25-29, 2013

Detecting Overlapping Speech with Long Short-Term Memory Recurrent Neural Networks

Jürgen T. Geiger (1), Florian Eyben (1), Björn Schuller (2), Gerhard Rigoll (1)

(1) Technische Universität München, Germany
(2) Universität Passau, Germany

Detecting segments of overlapping speech (when two or more speakers are active at the same time) is a challenging problem. Previously, mostly HMM-based systems have been used for overlap detection, employing various different audio features. In this work, we propose a novel overlap detection system using Long Short-Term Memory (LSTM) recurrent neural networks. LSTMs are used to generate framewise overlap predictions which are applied for overlap detection. Furthermore, a tandem HMM-LSTM system is obtained by adding LSTM predictions to the HMM feature set. Experiments with the AMI corpus show that overlap detection performance of LSTMs is comparable to HMMs. The combination of HMMs and LSTMs improves overlap detection by achieving higher recall.

Full Paper

Bibliographic reference.  Geiger, Jürgen T. / Eyben, Florian / Schuller, Björn / Rigoll, Gerhard (2013): "Detecting overlapping speech with long short-term memory recurrent neural networks", In INTERSPEECH-2013, 1668-1672.