Nearest neighbor pattern classification
Thomas M. Cover, Peter E. Hart · IEEE Transactions on Information Theory · 1967
The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points. This rule is independent of the underlying joint distribution on the sample points and their classifications, and hence the probability of errorRof such a rule must be at least as great as the Bayes probability of errorR^{\ast}--the minimum probability of error over all decision rules taking underlying probability structure into account. However, in a large sample analysis, we will show in theM-category case thatR^{\ast} \leq R \leq R^{\ast}(2 --MR^{\ast}/(M-1)), where these bounds are the tightest possible, for all suitably smooth underlying distributions. Thus for any number of categories, the probability of error of the nearest neighbor rule is bounded above by twice the Bayes probability of error. In this sense, it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.
Cited by 16,767 works. doi:10.1109/tit.1967.1053964
Authors
References
- The magical number seven, plus or minus two: Some limits on our capacity for processing information.
- Decision-Making Processes in Pattern Recognition
- A Nonparametric Estimate of a Multivariate Density Function
- The Information of Elementary Auditory Displays
- Applications of Information Theory to Psychology
- AN EMPIRICAL BAYES APPROACH TO NON-PARAMETRIC TWO-WAY CLASSIFICATION,
- Studies in Item Analysis and Prediction