Reference-Aware Language Models
Zichao Yang, Phil Blunsom, Chris Dyer et al. · Conference on Empirical Methods in Natural Language Processing (EMNLP) · 2017
We propose a general class of language models that treat reference as discrete stochastic latent variables.This decision allows for the creation of entity mentions by accessing external databases of referents (required by, e.g., dialogue generation) or past internal state (required to explicitly model coreferentiality).Beyond simple copying, our coreference model can additionally refer to a referent using varied mention forms (e.g., a reference to "Jane" can be realized as "she"), a characteristic feature of reference in natural languages.Experiments on three representative applications show our model variants outperform models based on deterministic attention and standard language modeling baselines.
Cited by 68 works. doi:10.18653/v1/d17-1197 Open-access PDF
Authors
References
- Long Short-Term Memory
- Neural Machine Translation by Jointly Learning to Align and Translate
- Recurrent neural network based language model
- Exploring the Limits of Language Modeling
- A Neural Conversational Model
- Addressing the Rare Word Problem in Neural Machine Translation
- Pointer networks
- Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
- Incorporating Copying Mechanism in Sequence-to-Sequence Learning
- A Neural Network Approach to Context-Sensitive Generation of Conversational Responses