Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding

Chen Liu, Su Zhu, Zijian Zhao, Ruisheng Cao, Lu Chen, Kai Yu


Spoken Language Understanding (SLU) converts hypotheses from automatic speech recognizer (ASR) into structured semantic representations. ASR recognition errors can severely degenerate the performance of the subsequent SLU module. To address this issue, word confusion networks (WCNs) have been used as the input for SLU, which contain richer information than 1-best or n-best hypotheses list. To further eliminate ambiguity, the last system act of dialogue context is also utilized as additional input. In this paper, a novel BERT based SLU model (WCN-BERT SLU) is proposed to encode WCNs and the dialogue context jointly. It can integrate both structural information and ASR posterior probabilities of WCNs in the BERT architecture. Experiments on DSTC2, a benchmark of SLU, show that the proposed method is effective and can outperform previous state-of-the-art models significantly.


 DOI: 10.21437/Interspeech.2020-1632

Cite as: Liu, C., Zhu, S., Zhao, Z., Cao, R., Chen, L., Yu, K. (2020) Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding. Proc. Interspeech 2020, 871-875, DOI: 10.21437/Interspeech.2020-1632.


@inproceedings{Liu2020,
  author={Chen Liu and Su Zhu and Zijian Zhao and Ruisheng Cao and Lu Chen and Kai Yu},
  title={{Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={871--875},
  doi={10.21437/Interspeech.2020-1632},
  url={http://dx.doi.org/10.21437/Interspeech.2020-1632}
}