This paper reports an experiment to assess the naturalness of the output of the prosodic rules in the CSTR ITS system. Both the duration rules and the intonation rules were evaluated. This work was carried out as part of the final evaluation of the text-to-speech software developed during the Alvey Integrated Speech Technology Demonstrator project: the other section of this evaluation is reported in Sydeserff et al. (1991). The experiment involved 165 subjects judging pairs of stimuli and simply indicating which was the more natural in a forced-choice design. The results clearly indicate that subjects could not reliably distinguish natural from automatic prosody, and that in the worst case our prosodic output is distinguishable from natural prosody in only 17% of cases.
Bibliographic reference. Monaghan, Alex I. C. (1991): "Evaluation of the naturalness of prosody generated by the CSTR TTS system", In EUROSPEECH-1991, 883-886.