Online dictionary learning for sparse coding
Julien Mairal, Francis Bach, Jean Ponce et al. · International Conference on Machine Learning · 2009
Sparse coding---that is, modelling data vectors as sparse linear combinations of basis elements---is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on learning the basis set, also called dictionary, to adapt it to specific data, an approach that has recently proven to be very effective for signal reconstruction and classification in the audio and image processing domains. This paper proposes a new online optimization algorithm for dictionary learning, based on stochastic approximations, which scales up gracefully to large datasets with millions of training samples. A proof of convergence is presented, along with experiments with natural images demonstrating that it leads to faster performance and better dictionaries than classical batch algorithms for both small and large datasets.
Cited by 2,114 works. doi:10.1145/1553374.1553463 Open-access PDF
Authors
References
- Regression Shrinkage and Selection Via the Lasso
- A Wavelet Tour of Signal Processing
- Nonlinear Programming
- Atomic Decomposition by Basis Pursuit
- Least angle regression
- Regularization and Variable Selection Via the Elastic Net
- A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics
- Sparse coding with an overcomplete basis set: A strategy employed by V1?
- $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
- Adaptive Algorithms and Stochastic Approximations