Recurrent Continuous Translation Models
Nal Kalchbrenner, Phil Blunsom · Conference on Empirical Methods in Natural Language Processing (EMNLP) · 2013
We introduce a class of probabilistic continuous translation models called Recurrent Continuous Translation Models that are purely based on continuous representations for words, phrases and sentences and do not rely on alignments or phrasal translation units.The models have a generation and a conditioning aspect.The generation of the translation is modelled with a target Recurrent Language Model, whereas the conditioning on the source sentence is modelled with a Convolutional Sentence Model.Through various experiments, we show first that our models obtain a perplexity with respect to gold translations that is > 43% lower than that of stateof-the-art alignment-based translation models.Secondly, we show that they are remarkably sensitive to the word order, syntax, and meaning of the source sentence despite lacking alignments.Finally we show that they match a state-of-the-art system when rescoring n-best lists of translations.
Cited by 1,232 works. doi:10.18653/v1/d13-1176 Open-access PDF
Authors
References
- Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
- A unified architecture for natural language processing
- The mathematics of statistical machine translation: parameter estimation
- Recurrent neural network based language model
- Semantic Compositionality through Recursive Matrix-Vector Spaces
- Generating Text with Recurrent Neural Networks
- Extensions of recurrent neural network language model
- Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection
- Continuous Space Translation Models for Phrase-Based Statistical Machine Translation
- Context dependent recurrent neural network language model