Exploring Deep Hybrid Tensor-to-Vector Network Architectures for Regression Based Speech Enhancement

Jun Qi, Hu Hu, Yannan Wang, Chao-Han Huck Yang, Sabato Marco Siniscalchi, Chin-Hui Lee


This paper investigates different trade-offs between the number of model parameters and enhanced speech qualities by employing several deep tensor-to-vector regression models for speech enhancement. We find that a hybrid architecture, namely CNN-TT, is capable of maintaining a good quality performance with a reduced model parameter size. CNN-TT is composed of several convolutional layers at the bottom for feature extraction to improve speech quality and a tensor-train (TT) output layer on the top to reduce model parameters. We first derive a new upper bound on the generalization power of the convolutional neural network (CNN) based vector-to-vector regression models. Then, we provide experimental evidence on the Edinburgh noisy speech corpus to demonstrate that, in single-channel speech enhancement, CNN outperforms DNN at the expense of a small increment of model sizes. Besides, CNN-TT slightly outperforms the CNN counterpart by utilizing only 32% of the CNN model parameters. Besides, further performance improvement can be attained if the number of CNN-TT parameters is increased to 44% of the CNN model size. Finally, our experiments of multi-channel speech enhancement on a simulated noisy WSJ0 corpus demonstrate that our proposed hybrid CNN-TT architecture achieves better results than both DNN and CNN models in terms of better-enhanced speech qualities and smaller parameter sizes.


 DOI: 10.21437/Interspeech.2020-1900

Cite as: Qi, J., Hu, H., Wang, Y., Yang, C.H., Siniscalchi, S.M., Lee, C. (2020) Exploring Deep Hybrid Tensor-to-Vector Network Architectures for Regression Based Speech Enhancement. Proc. Interspeech 2020, 76-80, DOI: 10.21437/Interspeech.2020-1900.


@inproceedings{Qi2020,
  author={Jun Qi and Hu Hu and Yannan Wang and Chao-Han Huck Yang and Sabato Marco Siniscalchi and Chin-Hui Lee},
  title={{Exploring Deep Hybrid Tensor-to-Vector Network Architectures for Regression Based Speech Enhancement}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={76--80},
  doi={10.21437/Interspeech.2020-1900},
  url={http://dx.doi.org/10.21437/Interspeech.2020-1900}
}