Bayesian Ensemble Learning
Hugh Chipman, Edward I. George, Robert E. McCulloch · The MIT Press eBooks · 2007
We develop a Bayesian “sum-of-trees” model, named BART, where each tree is constrained by a prior to be a weak learner. Fitting and inference are accomplished via an iterative backfitting MCMC algorithm. This model is motivated by ensemble methods in general, and boosting algorithms in particular. Like boosting, each weak learner (i.e., each weak tree) contributes a small amount to the overall model. However, our procedure is defined by a statistical model: a prior and a likelihood, while boosting is defined by an algorithm. This model-based approach enables a full and accurate assessment of uncertainty in model predictions, while remaining highly competitive in terms of predictive accuracy.
Cited by 97 works. doi:10.7551/mitpress/7503.003.0038 Open-access PDF
Authors
References
- Random Forests
- A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
- Greedy function approximation: A gradient boosting machine.
- Least angle regression
- Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination
- Detection of Influential Observation in Linear Regression
- Bayesian CART Model Search
- Bayesian backfitting (with comments and a rejoinder by the authors
- Bayesian CART: Prior Specification and Posterior Simulation
- Visualizable and interpretable regression models with good prediction power