Auditory-Visual Speech Processing 2005
British Columbia, Canada
Motivated by the success of free-parts based representations in face recognition  we have attempted to address some of the problems associated with applying such a philosophy to the task of speaker-independent automatic speech reading. Hitherto, a major problem with canonical area-based approaches in automatic speech reading is the intrinsic lack of training observations due to the visual speech modality's low sample rate and large variability in appearance. We believe a free-parts representation can overcome many of these limitations due to its natural ability to generalize by producing many observations from a single mouth image, whilst still preserving the ability to discriminate between various visual-speech units. This approach additionally requires a modification to traditional techniques employed for the estimation of hidden Markov Models (HMMs), whose resultant models we currently refer to as free-parts HMMs (FP-HMMs). Results will be presented on the CUAVE audio-visual speech database.
Bibliographic reference. Lucey, Simon / Lucey, Patrick (2005): "Improved speech reading through a free-parts representation", In AVSP-2005, 85-86.