Domain Adaptation for Statistical Classifiers
Hal Daumé, Daniel Marcu · Journal of Artificial Intelligence Research · 2006
The most basic assumption used in statistical learning theory is that training data and test data are drawn from the same underlying distribution. Unfortunately, in many applications, the "in-domain" test data is drawn from a distribution that is related, but not identical, to the "out-of-domain" distribution of the training data. We consider the common case in which labeled out-of-domain data is plentiful, but labeled in-domain data is scarce. We introduce a statistical formulation of this problem in terms of a simple mixture model and present an instantiation of this framework to maximum entropy classifiers and their linear chain counterparts. We present efficient inference algorithms for this special case based on the technique of conditional expectation maximization. Our experimental results show that our approach leads to improved performance on three real world tasks on four different data sets from the natural language processing domain.
Cited by 816 works. doi:10.1613/jair.1872 Open-access PDF
Authors
References
- Maximum Likelihood from Incomplete Data Via the EM Algorithm
- Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
- Multitask Learning
- Inducing features of random fields
- Hierarchical Mixtures of Experts and the EM Algorithm
- Maximum Entropy Markov Models for Information Extraction and Segmentation
- Nonparametric Statistical Inference
- An extensive empirical study of feature selection metrics for text classification
- A comparison of algorithms for maximum entropy parameter estimation
- Is Learning The n-th Thing Any Easier Than Learning The First?