Auditory-Visual Speech Processing (AVSP) 2010

Hakone, Kanagawa, Japan
September 30-October 3, 2010

In Pursuit of Visemes

Sarah Hilder, Barry-John Theobald, Richard Harvey

School of Computing Sciences, University of East Anglia, UK

We describe preliminary work towards an objective method for identifying visemes. Active appearance model (AAM) features are used to parameterise a speaker’s lips and jaw during speech. The temporal behaviour of AAM features between automatically identified salient points is used to represent visual speech gestures, and visemes are created by clustering these gestures using dynamic time warping (DTW) as a costfunction. This method produces a significantly more structured model of visual speech than if a typical phoneme-to-viseme mapping is assumed.

Index Terms: Visemes, visual speech encoding

Full Paper

Bibliographic reference.  Hilder, Sarah / Theobald, Barry-John / Harvey, Richard (2010): "In pursuit of visemes", In AVSP-2010, paper S8-2.