First International Conference on Spoken Language Processing (ICSLP 90)

Kobe, Japan
November 18-22, 1990

Evaluating Synthesiser Performance: Is Segmental Intelligibility Enough?

Kim Silverman, Sara Basson, Suzi Levas

NYNEX Science and Technology, White Plains, NY, USA

Laboratory-based evaluations of synthetic speech often suggest that it is as intelligible as natural speech. Yet studies and experience in applied settings do not confirm this. We identify two issues underlying this discrepancy: (i) neither the speech material nor the listeners' task in typical evaluations sufficiently represent application conditions, and (ii) tests should measure the cognitive load accompanying a given intelligibility score. Our preliminary data for two commercial synthesisers underline both issues: the two synthesisers were ranked in one order on a segmental intelligibility test, but in the opposite order on a more application-like comprehension test. For the latter test, listeners' accuracy may be related to their performance speed. Results are related to the underlying issues, and two approaches to measurement of cognitive load are suggested.

Full Paper

Bibliographic reference.  Silverman, Kim / Basson, Sara / Levas, Suzi (1990): "Evaluating synthesiser performance: is segmental intelligibility enough?", In ICSLP-1990, 981-984.