ISCA Archive Interspeech 2013
ISCA Archive Interspeech 2013

Recurrent neural networks for language understanding

Kaisheng Yao, Geoffrey Zweig, Mei-Yuh Hwang, Yangyang Shi, Dong Yu

Recurrent Neural Network Language Models (RNN-LMs) have recently shown exceptional performance across a variety of applications. In this paper, we modify the architecture to perform Language Understanding, and advance the state-of-the-art for the widely used ATIS dataset. The core of our approach is to take words as input as in a standard RNN-LM, and then to predict slot labels rather than words on the output side. We present several variations that differ in the amount of word context that is used on the input side, and in the use of non-lexical features. Remarkably, our simplest model produces state-of-the-art results, and we advance state-of-the-art through the use of bag-of-words, word embedding, named-entity, syntactic, and word-class features. Analysis indicates that the superior performance is attributable to the task-specific word representations learned by the RNN.

doi: 10.21437/Interspeech.2013-569

Cite as: Yao, K., Zweig, G., Hwang, M.-Y., Shi, Y., Yu, D. (2013) Recurrent neural networks for language understanding. Proc. Interspeech 2013, 2524-2528, doi: 10.21437/Interspeech.2013-569

  author={Kaisheng Yao and Geoffrey Zweig and Mei-Yuh Hwang and Yangyang Shi and Dong Yu},
  title={{Recurrent neural networks for language understanding}},
  booktitle={Proc. Interspeech 2013},