EUROSPEECH 2001 Scandinavia
7th European Conference on Speech Communication and Technology

Aalborg, Denmark
September 3-7, 2001


An Investigation of HMM Classifier Combination Strategies for Improved Audio-Visual Speech Recognition

Simon Lucey, Sridha Sridharan, Vinod Chandran

Queensland University of Technology, Australia

The combining of independent audio and visual HMM classifiers (late integration) has been shown to out perform the combination of audio and visual features in a single HMM classifier (early integration) when either or both modalities are presented with distortion for the task of speech recognition. Theoretical foundations for the optimal combination of these audio and video classifiers are still unclear. In this paper a number of strategies for combining these classifiers are investigated. An argument for using a hybrid of the sum and product rules is made based on empirical, theoretical and heuristic evidence.

Full Paper

Bibliographic reference.  Lucey, Simon / Sridharan, Sridha / Chandran, Vinod (2001): "An investigation of HMM classifier combination strategies for improved audio-visual speech recognition", In EUROSPEECH-2001, 1185-1188.