EUROSPEECH 2001 Scandinavia
The combining of independent audio and visual HMM classifiers (late integration) has been shown to out perform the combination of audio and visual features in a single HMM classifier (early integration) when either or both modalities are presented with distortion for the task of speech recognition. Theoretical foundations for the optimal combination of these audio and video classifiers are still unclear. In this paper a number of strategies for combining these classifiers are investigated. An argument for using a hybrid of the sum and product rules is made based on empirical, theoretical and heuristic evidence.
Bibliographic reference. Lucey, Simon / Sridharan, Sridha / Chandran, Vinod (2001): "An investigation of HMM classifier combination strategies for improved audio-visual speech recognition", In EUROSPEECH-2001, 1185-1188.