Objective Quality Assessment of Target Speaker Separation Performance in Multisource Reverberant Environment

Paco Langjahr, Pejman Mowlaee


Sound quality estimation of a speech enhancement or source separation system in a realistic adverse noise scenario is a challenge. In particular, the connection between results obtained by quality metrics versus those obtained from human subjective listening tests is unknown. In this paper, as the first attempt, we present results, which examine the correlation between certain objective metrics and Multi Stimulus with Hidden Reference and Anchor Point (MUSHRA) listening tests. There are a variety of applications which may benefit from the outcome of this study, since the enhancement/separation system is often used as a pre-processing stage for target applications such as - to name a few - automatic speech recognition, speech coding, speaker recognition and hearing aid devices. In all these applications the knowledge of which objective metrics reliably predicts subjective results is of great importance, as it saves the time consuming task of listening tests in future studies when developing a new enhancement system. The paper presents performance evaluation of several benchmark systems in a realistic noisy reverberant environment using the existing objective quality metrics. Subjective listening experiments which are relevant for perceptual speech quality. MUSHRA listening test is also conducted as our subjective evaluation. Comparing the objective and subjective results we present the discussion about which metrics correlate well with subjective listening tests.


 DOI: 10.21437/PQS.2013-16

Cite as: Langjahr, P., Mowlaee, P. (2013) Objective Quality Assessment of Target Speaker Separation Performance in Multisource Reverberant Environment. Proc. 4th International Workshop on Perceptual Quality of Systems (PQS 2013), 79-84, DOI: 10.21437/PQS.2013-16.


@inproceedings{Langjahr2013,
  author={Paco Langjahr and Pejman Mowlaee},
  title={Objective Quality Assessment of Target Speaker Separation Performance in Multisource Reverberant Environment},
  year=2013,
  booktitle={Proc. 4th International Workshop on Perceptual Quality of Systems (PQS 2013)},
  pages={79--84},
  doi={10.21437/PQS.2013-16},
  url={http://dx.doi.org/10.21437/PQS.2013-16}
}