13th Annual Conference of the International Speech Communication Association

Portland, OR, USA
September 9-13, 2012

Speech Factorization for HMM-TTS Based on Cluster Adaptive Training

Javier Latorre, Vincent Wan, Mark J. F. Gales, Langzhou Chen, K. K. Chin, Kate Knill, Masami Akamine

Toshiba Research Europe Ltd., Cambridge, UK

This paper presents a novel approach to factorize and control different speech factors in HMM-based TTS systems. In this paper cluster adaptive training (CAT) is used to factorize speaker identity and expressiveness (i.e. emotion). Within a CAT framework, each speech factor can be modelled by a different set of clusters. Users can control speaker identity and expressiveness independently by modifying the weights associated with each set. These weights are defined in a continuous space, so variations of speaker and emotion are also continuous. Additionally, given a speaker which has only neutral-style training data, the approach is able to synthesise speech with that speaker's voice and different expressions. Lastly, the paper discusses how generalization of the basic factorization concept could allow the production of expressive speech from neutral voices for other HMM-TTS systems not based on CAT.

Index Terms: speech synthesis, cluster adaptive training, expressive synthesis, speech factorization

Full Paper

Bibliographic reference.  Latorre, Javier / Wan, Vincent / Gales, Mark J. F. / Chen, Langzhou / Chin, K. K. / Knill, Kate / Akamine, Masami (2012): "Speech factorization for HMM-TTS based on cluster adaptive training", In INTERSPEECH-2012, 971-974.