Auditory-Visual Speech Processing (AVSP) 2013

Annecy, France
August 29 - September 1, 2013

Visual Front-End Wars: Viola-Jones Face Detector vs Fourier Lucas-Kanade

Shahram Kalantari, Rajitha Navarathna, David Dean, Sridha Sridharan

Speech, Audio, Image and Video Technology Lab, Queensland University of Technology, Australia.

The performance of visual speech recognition (VSR) systems are significantly influenced by the accuracy of the visual front-end. The current state-of-the-art VSR systems use off-the-shelf face detectors such as Viola- Jones (VJ) which has limited reliability for changes in illumination and head poses. For a VSR system to perform well under these conditions, an accurate visual front end is required. This is an important problem to be solved in many practical implementations of audio visual speech recognition systems, for example in automotive environments for an efficient human-vehicle computer interface. In this paper, we re-examine the current state-of-the-art VSR by comparing off-the-shelf face detectors with the recently developed Fourier Lucas-Kanade (FLK) image alignment technique. A variety of image alignment and visual speech recognition experiments are performed on a clean dataset as well as with a challenging automotive audio-visual speech dataset. Our results indicate that the FLK image alignment technique can significantly outperform off-the shelf face detectors, but requires frequent fine-tuning.

Index Terms: Visual Front-ends, Viola-Jones, Fourier Lucas-Kanade, Visual Speech Recognition

Full Paper

Bibliographic reference.  Kalantari, Shahram / Navarathna, Rajitha / Dean, David / Sridharan, Sridha (2013): "Visual front-endwars: Viola-Jones face detector vs Fourier Lucas-Kanade", In AVSP-2013, 163-168.