Solving large scale linear prediction problems using stochastic gradient descent algorithms
Tong Zhang · 2004
Linear prediction methods, such as least squares for regression, logistic regression and support vector machines for classification, have been extensively used in statistics and machine learning. In this paper, we study stochastic gradient descent (SGD) algorithms on regularized forms of linear prediction methods. This class of methods, related to online algorithms such as perceptron, are both efficient and very simple to implement. We obtain numerical rate of convergence for such algorithms, and discuss its implications. Experiments on text data will be provided to demonstrate numerical and statistical consequences of our theoretical findings.
Cited by 1,106 works. doi:10.1145/1015330.1015332
Authors
References
- Acceleration of Stochastic Approximation by Averaging
- Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms
- Discriminative training methods for hidden Markov models
- Large margin classification using the perceptron algorithm
- Stochastic Approximation Algorithms and Applications
- Text Categorization Based on Regularized Linear Classification Methods
- Relative Loss Bounds for Multidimensional Regression Problems
- Analysis of Two Gradient-Based Algorithms for On-Line Regression
- A loss function analysis for classification methods in text categorization
- Large Margin Classification for Moving Targets