AUC Optimization vs. Error Rate Minimization
Corinna Cortes, Mehryar Mohri · 2003
The area under an ROC curve (AUC) is a criterion used in many applications to measure the quality of a classification algorithm. However, the objective function optimized in most of these algorithms is the error rate and not the AUC value. We give a detailed statistical analysis of the relationship between the AUC and the error rate, including the first exact expression of the expected value and the variance of the AUC for a fixed error rate. Our results show that the average AUC is monotonically increasing as a function of the classification accuracy, but that the standard deviation for uneven distributions and higher error rates is noticeable. Thus, algorithms designed to minimize the error rate may not lead to the best possible AUC values. We show that, under certain conditions, the global function optimized by the RankBoost algorithm is exactly the AUC. We report the results of our experiments with RankBoost in several datasets demonstrating the benefits of an algorithm specifically designed to globally optimize the AUC over other existing algorithms optimizing an approximation of the AUC or only locally optimizing the AUC. 1
Cited by 498 works.
Authors
References
- Classification and Regression Trees.
- Classification and Regression Trees.
- The meaning and use of the area under a receiver operating characteristic (ROC) curve.
- A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
- Signal detection theory and psychophysics
- Signal detection theory and ROC analysis
- Analysis and visualization of classifier performance: comparison under imprecise class and cost distributions
- Learning Decision Trees Using the Area Under the ROC Curve
- 10.1162/1532443041827916
- 10.1162/153244304322972694