Bootstrap Techniques for Error Estimation
Anil Kumar Jain, Richard C. Dubes, Chien-Chang Chen · IEEE Transactions on Pattern Analysis and Machine Intelligence · 1987
The design of a pattern recognition system requires careful attention to error estimation. The error rate is the most important descriptor of a classifier's performance. The commonly used estimates of error rate are based on the holdout method, the resubstitution method, and the leave-one-out method. All suffer either from large bias or large variance and their sample distributions are not known. Bootstrapping refers to a class of procedures that resample given data by computer. It permits determining the statistical properties of an estimator when very little is known about the underlying distribution and no additional samples are available. Since its publication in the last decade, the bootstrap technique has been successfully applied to many statistical estimations and inference problems. However, it has not been exploited in the design of pattern recognition systems. We report results on the application of several bootstrap techniques in estimating the error rate of 1-NN and quadratic classifiers. Our experiments show that, in most cases, the confidence interval of a bootstrap estimator of classification error is smaller than that of the leave-one-out estimator. The error of 1-NN, quadratic, and Fisher classifiers are estimated for several real data sets.
Cited by 205 works. doi:10.1109/tpami.1987.4767957
Authors
References
- Pattern classification and scene analysis
- THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS
- Bootstrap Methods: Another Look at the Jackknife
- The Jackknife, the Bootstrap and Other Resampling Plans
- Multivariate Statistical Methods
- Pattern Classification and Scene Analysis.
- Pattern recognition : a statistical approach
- A Leisurely Look at the Bootstrap, the Jackknife, and Cross-Validation
- Estimation of Error Rates in Discriminant Analysis
- Computer-Intensive Methods in Statistics