Improving Machine Translation Performance by Exploiting Non-Parallel Corpora
Dragos Stefan Munteanu, Daniel Marcu · Computational Linguistics · 2005
We present a novel method for discovering parallel sentences in comparable, non-parallel corpora. We train a maximum entropy classifier that, given a pair of sentences, can reliably determine whether or not they are translations of each other. Using this approach, we extract parallel data from large Chinese, Arabic, and English non-parallel newspaper corpora. We evaluate the quality of the extracted data by showing that it improves the performance of a state-of-the-art statistical machine translation system. We also show that a good-quality MT system can be built from scratch by starting with a very small parallel corpus (100,000 words) and exploiting a large non-parallel corpus. Thus, our method can be applied with great benefit to language pairs for which only scarce resources are available.
Cited by 391 works. doi:10.1162/089120105775299168 Open-access PDF
Authors
References
- BLEU
- The mathematics of statistical machine translation: parameter estimation
- A statistical approach to machine translation
- Generalized Iterative Scaling for Log-Linear Models
- A Systematic Comparison of Various Statistical Alignment Models
- The Annals of Mathematical Statistics
- Minimum error rate training in statistical machine translation
- A program for aligning sentences in bilingual corpora
- Some Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval
- A comparison of algorithms for maximum entropy parameter estimation