PAC-Bayesian model averaging
David McAllester · 1999
PAC-Bayesian learning methods combine the informative priors of Bayesian methods with distribution-free PAC guarantees. Building on earlier methods for PAC-Bayesian model selection, this paper presents a method for PAC-Bayesian model averaging. The main result is a bound on generalization error of an arbitrary weighted mixture of concepts that depends on the empirical error of that mixture and the KLdivergence of the mixture from the prior. A simple characterization is also given for the error bound achieved by the optimal weighting. 1
Cited by 354 works. doi:10.1145/307400.307435
Authors
References
- Learning classification trees
- Minimum complexity density estimation
- An experimental and theoretical comparison of model selection methods
- Complexity Regularization with Application to Artificial Neural Networks
- On Pruning and Averaging Decision Trees
- Predicting nearly as well as the best pruning of a decision tree
- Some PAC-Bayesian theorems
- A PAC analysis of a Bayesian estimator
- Concept learning using complexity regularization
- A Fast, Bottom-Up Decision Tree Pruning Algorithm with Near-Optimal Generalization