Self-taught learning
Rajat Raina, Alexis J. Battle, Honglak Lee et al. · International Conference on Machine Learning · 2007
We present a new machine learning framework called "self-taught learning" for using unlabeled data in supervised classification tasks. We do not assume that the unlabeled data follows the same class labels or generative distribution as the labeled data. Thus, we would like to use a large number of unlabeled images (or audio samples, or text documents) randomly downloaded from the Internet to improve performance on a given image (or audio, or text) classification task. Such unlabeled data is significantly easier to obtain than in typical semi-supervised or transfer learning settings, making self-taught learning widely applicable to many practical learning problems. We describe an approach to self-taught learning that uses sparse coding to construct higher-level features using the unlabeled data. These features form a succinct input representation and significantly improve classification performance. When using an SVM for classification, we further show how a Fisher kernel can be learned for this representation.
Cited by 1,510 works. doi:10.1145/1273496.1273592
Authors
References
- Reducing the Dimensionality of Data with Neural Networks
- Regression Shrinkage and Selection Via the Lasso
- Indexing by latent semantic analysis
- Latent dirichlet allocation
- Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
- Nonlinear Dimensionality Reduction by Locally Linear Embedding
- A Global Geometric Framework for Nonlinear Dimensionality Reduction
- Least angle regression
- Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories
- Emergence of simple-cell receptive field properties by learning a sparse code for natural images