Statistical phrase-based translation
Philipp Koehn, Franz Josef Och, Daniel Marcu · 2003
We propose a new phrase-based translation model and decoding algorithm that enables us to evaluate and compare several, previously proposed phrase-based translation models. Within our framework, we carry out a large number of experiments to understand better and explain why phrase-based models out-perform word-based models. Our empirical results, which hold for all examined language pairs, suggest that the highest levels of performance can be obtained through relatively simple means: heuristic learning of phrase translations from word-based alignments and lexical weighting of phrase translations. Surprisingly, learning phrases longer than three words and learning phrases from high-accuracy word-level alignment models does not have a strong impact on performance. Learning only syntactically motivated phrases degrades the performance of our systems.
Cited by 3,197 works. doi:10.3115/1073445.1073462 Open-access PDF
Authors
References
- BLEU
- The mathematics of statistical machine translation: parameter estimation
- Statistical Methods for Speech Recognition
- Three generative, lexicalised models for statistical parsing
- Statistical language modeling using the CMU-cambridge toolkit
- Improved statistical alignment models
- Improved Alignment Models for Statistical Machine Translation
- A phrase-based, joint probability model for statistical machine translation
- A syntax-based statistical translation model
- Stochastic inversion transduction grammars and bilingual parsing of parallel corpora