High-dimensional classification using features annealed independence rules
Jianqing Fan, Yingying Fan · The Annals of Statistics · 2008
Classification using high-dimensional features arises frequently in many contemporary statistical studies such as tumor classification using microarray or other high-throughput data. The impact of dimensionality on classifications is largely poorly understood. In a seminal paper, Bickel and Levina (2004) show that the Fisher discriminant performs poorly due to diverging spectra and they propose to use the independence rule to overcome the problem. We first demonstrate that even for the independence classification rule, classification using all the features can be as bad as the random guessing due to noise accumulation in estimating population centroids in high-dimensional feature space. In fact, we demonstrate further that almost all linear discriminants can perform as bad as the random guessing. Thus, it is paramountly important to select a subset of important features for high-dimensional classification, resulting in Features Annealed Independence Rules (FAIR). The conditions under which all the important features can be selected by the two-sample t-statistic are established. The choice of the optimal number of features, or equivalently, the threshold value of the test statistics are proposed based on an upper bound of the classification error. Simulation studies and real data analysis support our theoretical results and demonstrate convincingly the advantage of our new classification procedure.
Cited by 480 works. doi:10.1214/07-aos504 Open-access PDF
Authors
References
- Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring
- Regularized Discriminant Analysis
- Weak Convergence and Empirical Processes
- Diagnosis of multiple cancer types by shrunken centroids of gene expression
- Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data
- Predicting the clinical status of human breast cancer by using gene expression profiles
- Gene expression correlates of clinical prostate cancer behavior
- Sparse Principal Component Analysis
- Sure Independence Screening for Ultrahigh Dimensional Feature Space
- Persistence in high-dimensional linear predictor selection and the virtue of overparametrization