Boosting Decision Trees
Harris Drucker, Corinna Cortes · neural information processing systems · 1995
We introduce a constructive, incremental learning system for regression problems that models data by means of locally linear experts. In contrast to other approaches, the experts are trained independently and do not compete for data during learning. Only when a prediction for a query is required do the experts cooperate by blending their individual predictions. Each expert is trained by minimizing a penalized local cross validation error using second order methods. In this way, an expert is able to find a local distance metric by adjusting the size and shape of the receptive field in which its predictions are valid, and also to detect relevant input features by adjusting its bias on the importance of individual input dimensions. We derive asymptotic results for our method. In a variety of simulations the properties of the algorithm are demonstrated with respect to interference, learning speed, prediction accuracy, feature detection, and task oriented incremental learning.
Cited by 220 works.
Authors
References
- Classification and Regression Trees.
- Bagging Predictors
- A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
- The Strength of Weak Learnability
- Comparison of classifier methods: a case study in handwritten digit recognition
- Multiple decision trees
- BOOSTING PERFORMANCE IN NEURAL NETWORKS
- An Empirical Comparison of Pruning Methods for Decision Tree Induction
- Boosting and Other Ensemble Methods
- Boosting a weak learning algorithm by majority