Conditional variable importance for random forests
Carolin Strobl, Anne‐Laure Boulesteix, Thomas Kneib et al. · BMC Bioinformatics · 2008
BACKGROUND: Random forests are becoming increasingly popular in many scientific fields because they can cope with "small n large p" problems, complex interactions and even highly correlated predictor variables. Their variable importance measures have recently been suggested as screening tools for, e.g., gene expression studies. However, these variable importance measures show a bias towards correlated predictor variables. RESULTS: We identify two mechanisms responsible for this finding: (i) A preference for the selection of correlated predictors in the tree building process and (ii) an additional advantage for correlated predictor variables induced by the unconditional permutation scheme that is employed in the computation of the variable importance measure. Based on these considerations we develop a new, conditional permutation scheme for the computation of the variable importance measure. CONCLUSION: The resulting conditional variable importance reflects the true impact of each predictor variable more reliably than the original marginal approach.
Cited by 3,254 works. doi:10.1186/1471-2105-9-307 Open-access PDF
Authors
References
- R: A Language and Environment for Statistical Computing
- Random Forests
- Classification and Regression Trees.
- Classification and Regression Trees.
- Bagging Predictors
- The Elements of Statistical Learning
- An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants
- An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization
- Arcing classifier (with discussion and a rejoinder by the author)
- Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)