Self-Paced Learning for Latent Variable Models
Manish Kumar, Benjamin Packer, Daphne Koller · neural information processing systems · 2010
Latent variable models are a powerful tool for addressing several tasks in machine learning. However, the algorithms for learning the parameters of latent variable models are prone to getting stuck in a bad local optimum. To alleviate this problem, we build on the intuition that, rather than considering all samples simultaneously, the algorithm should be presented with the training data in a meaningful order that facilitates learning. The order of the samples is determined by how easy they are. The main challenge is that often we are not provided with a readily computable measure of the easiness of samples. We address this issue by proposing a novel, iterative self-paced learning algorithm where each iteration simultaneously selects easy samples and learns a new parameter vector. The number of samples selected is governed by a weight that is annealed until the entire training data has been considered. We empirically demonstrate that the self-paced learning algorithm outperforms the state of the art method for learning a latent structural SVM on four applications: object localization, noun phrase coreference, motif finding and handwritten digit recognition.
Cited by 969 works.
Authors
References
- Maximum Likelihood from Incomplete Data Via the EM Algorithm
- Gradient-based learning applied to document recognition
- Histograms of Oriented Gradients for Human Detection
- Bayesian Data Analysis
- A discriminatively trained, multiscale, deformable part model
- Combining labeled and unlabeled data with co-training
- Nonlinear programming: theory and algorithms
- Curriculum learning
- Tangent Prop - A formalism for specifying selected invariances in an adaptive network
- Max-Margin Markov Networks