BOOSTING ALGORITHMS: REGULARIZATION, PREDICTION AND MODEL FITTING
Peter Bühlmann, Torsten Hothorn · 2007
Abstract. We present a statistical perspective on boosting. Special emphasis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as well as regression models for survival analysis. Concepts of degrees of freedom and corresponding Akaike or Bayesian information criteria, particularly useful for regularization and variable selection in highdimensional covariate spaces, are discussed as well. The practical aspects of boosting procedures for fitting statistical models are illustrated by means of the dedicated open-source software package mboost. This package implements functions which can be used for model fitting, prediction and variable selection. It is flexible, allowing for the implementation of new boosting algorithms optimizing user-specified loss functions. Key words and phrases: Generalized linear models, generalized additive models, gradient boosting, survival analysis, variable selection, software. 1.
Cited by 685 works.
Authors
References
- Random Forests
- Bagging Predictors
- Regression Shrinkage and Selection Via the Lasso
- A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
- Experiments with a new boosting algorithm
- Exploratory Data Analysis
- UCI Repository of machine learning databases
- Greedy function approximation: A gradient boosting machine.
- The Elements of Statistical Learning: Data Mining, Inference, and Prediction
- Matching pursuits with time-frequency dictionaries