Improved Learning of Word Embeddings with Word Definitions and Semantic Injection

Yichi Zhang, Yinpei Dai, Zhijian Ou, Huixin Wang, Junlan Feng


Recently, two categories of linguistic knowledge sources, word definitions from monolingual dictionaries and linguistic relations (e.g. synonymy and antonymy), have been leveraged separately to improve the traditional co-occurrence based methods for learning word embeddings. In this paper, we investigate to leverage these two kinds of resources together. Specifically, we propose a new method for word embedding specialization, named Definition Autoencoder with Semantic Injection (DASI). In our experiments1, DASI outperforms its single-knowledge-source counterparts on two semantic similarity benchmarks, and the improvements are further justified on a downstream task of dialog state tracking. We also show that DASI is superior over simple combinations of existing methods in incorporating the two knowledge sources.


 DOI: 10.21437/Interspeech.2020-1702

Cite as: Zhang, Y., Dai, Y., Ou, Z., Wang, H., Feng, J. (2020) Improved Learning of Word Embeddings with Word Definitions and Semantic Injection. Proc. Interspeech 2020, 4253-4257, DOI: 10.21437/Interspeech.2020-1702.


@inproceedings{Zhang2020,
  author={Yichi Zhang and Yinpei Dai and Zhijian Ou and Huixin Wang and Junlan Feng},
  title={{Improved Learning of Word Embeddings with Word Definitions and Semantic Injection}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={4253--4257},
  doi={10.21437/Interspeech.2020-1702},
  url={http://dx.doi.org/10.21437/Interspeech.2020-1702}
}