Improved boosting algorithms using confidence-rated predictions
Robert E. Schapire, Yoram Singer · 1998
. We describe several improvements to Freund and Schapire's AdaBoost boosting algorithm, particularly in a setting in which hypotheses may assign confidences to each of their predictions. We give a simplified analysis of AdaBoost in this setting, and we show how this analysis can be used to find improved parameter settings as well as a refined criterion for training weak hypotheses. We give a specific method for assigning confidences to the predictions of decision trees, a method closely related to one used by Quinlan. This method also suggests a technique for growing decision trees which turns out to be identical to one proposed by Kearns and Mansour. We focus next on how to apply the new boosting algorithms to multiclass classification problems, particularly to the multi-label case in which each example may belong to more than one class. We give two boosting methods for this problem, plus a third method based on output coding. One of these leads to a new method for handling the singl...
Cited by 2,547 works. doi:10.1145/279943.279960
Authors
References
- A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
- Experiments with a new boosting algorithm
- Boosting the margin: a new explanation for the effectiveness of voting methods
- Advances in neural information processing systems 7
- What Size Net Gives Valid Generalization?
- An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants
- An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization
- Solving Multiclass Learning Problems via Error-Correcting Output Codes
- Arcing classifier (with discussion and a rejoinder by the author)
- UCI Repository of Machine Learning Databases