Variable Selection Bias in Classification Trees Based on Imprecise Probabilities
Carolin Strobl · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · 2005
Classification trees based on imprecise probabilities provide an advancement of classical classification trees. The Gini Index is the default splitting criterion in classical classification trees, while in classification trees based on imprecise probabilities, an extension of the Shannon entropy has been introduced as the splitting criterion. However, the use of these empirical entropy measures as split selection criteria can lead to a bias in variable selection, such that variables are preferred for features other than their information content. This bias is not eliminated by the imprecise probability approach. The source of variable selection bias for the estimated Shannon entropy, as well as possible corrections, are outlined. The variable selection performance of the biased and corrected estimators are evaluated in a simulation study. Additional results from research on variable selection bias in classical classification trees are incorporated, implying further investigation of alternative split selection criteria in classification trees based on imprecise probabilities.
Cited by 10 works. doi:10.5282/ubm/epub.1788 Open-access PDF
Authors
References
- A Mathematical Theory of Communication
- On a Statistical Estimate for the Entropy of a Sequence of Independent Random Variables
- Classification Trees With Unbiased Multiway Splits
- Estimating the errors on measured entropy and mutual information
- Bias analysis in entropy estimation
- MAXIMUM OF ENTROPY FOR CREDAL SETS
- Upper entropy of credal sets. Applications to credal classification
- Statistical Sources of Variable Selection Bias in Classification Tree Algorithms Based on the Gini Index
- An Update on Generalized Information Theory.
- Uncertainty and Information Measures for Imprecise Probabilities: An Overview.