The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
k-Nearest Neighbour Classifiers - A Tutorial
Open accessAbstract
Perhaps the most straightforward classifier in the arsenal or Machine Learning techniques is the Nearest Neighbour Classifier—classification is achieved by identifying the nearest neighbours to a query example and using those neighbours to determine the class of the query. This approach to classification is of particular importance, because issues of poor runtime performance is not such a problem these days with the computational power that is available. This article presents an overview of techniques for Nearest Neighbour classification focusing on: mechanisms for assessing similarity (distance), computational issues in identifying nearest neighbours, and mechanisms for reducing the dimension of the data. This article is the second edition of a paper previously published as a technical report [16]. Sections on similarity measures for time-series, retrieval speedup, and intrinsic dimensionality have been added. An Appendix is included, providing access to Python code for the key methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1992) · Cited by 23,431
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
Wrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,947
Multidimensional binary search trees used for associative searching
Bentley, J. L. (1975) · Communications of the ACM · doi:10.1145/361002.361007 · Cited by 7,520
An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression
Altman, N. S. (1992) · The American Statistician · doi:10.1080/00031305.1992.10475879 · Cited by 5,708
The Earth Mover's Distance as a Metric for Image Retrieval
Rubner, Y., Tomasi, C., & Guibas, L. J. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026543900054 · Cited by 4,568
Approximate nearest neighbors: towards removing the curse of dimensionality
Indyk, P., & Motwani, R. (1998) · Proceedings of the thirtieth annual ACM symposium on Theory of computing - STOC '98 · doi:10.1145/276698.276876 · Cited by 4,188
2009 International Joint Conference on Artificial Intelligence
(2009) · doi:10.1109/jcai15434.2009 · Cited by 2,800
Instance-based learning algorithms
Aha, D. W., Kibler, D., & Albert, M. K. (1991) · Machine Learning · doi:10.1007/bf00153759 · Cited by 2,163
Asymptotic Properties of Nearest Neighbor Rules Using Edited Data
Wilson, D. L. (1972) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1972.4309137 · Cited by 2,130
A symbolic representation of time series, with implications for streaming algorithms
Lin, J., Keogh, E., Lonardi, S., & Chiu, B. (2003) · Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery · doi:10.1145/882082.882086 · Cited by 1,799
The condensed nearest neighbor rule (Corresp.)
Hart, P. (1968) · IEEE Transactions on Information Theory · doi:10.1109/tit.1968.1054155 · Cited by 1,733
Nearest neighbor (NN) norms: NN pattern classification techniques
Dasarathy, B. V. (1991) · Medical Entomology and Zoology · Cited by 1,654
Bagnall, A., Lines, J., Bostrom, A., Large, J., & Keogh, E. (2017) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-016-0483-9 · Cited by 1,377
de Winter, J. C. F., Gosling, S. D., & Potter, J. (2016) · Psychological Methods · doi:10.1037/met0000079 · Cited by 1,201
Reduction Techniques for Instance-Based Learning Algorithms
Wilson, D. R., & Martinez, T. R. (2000) · Machine Learning · doi:10.1023/a:1007626913721 · Cited by 1,182
Derivative Dynamic Time Warping
Keogh, E. J., & Pazzani, M. J. (2001) · Proceedings of the 2001 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972719.1 · Cited by 1,143
Li, M., Chen, X., Li, X., Ma, B., & Vitanyi, P. M. B. (2004) · IEEE Transactions on Information Theory · doi:10.1109/tit.2004.838101 · Cited by 1,062
Cover trees for nearest neighbor
Beygelzimer, A., Kakade, S., & Langford, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143857 · Cited by 813
A Branch and Bound Algorithm for Computing k-Nearest Neighbors
Fukunaga, K., & Narendra, P. M. (1975) · IEEE Transactions on Computers · doi:10.1109/t-c.1975.224297 · Cited by 736
Prototype Selection for Nearest Neighbor Classification: Taxonomy and Empirical Study
Garcia, S., Derrac, J., Cano, J. R., & Herrera, F. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.142 · Cited by 695
Random search for hyper-parameter optimization
BergstraJames, & BengioYoshua (2012) · Journal of Machine Learning Research · doi:10.5555/2188385.2188395 · Cited by 690
Optimised KD-trees for fast image descriptor matching
Silpa-Anan, C., & Hartley, R. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587638 · Cited by 677
A symbolic representation of time series, with implications for streaming algorithms
Lin, J., Keogh, E., Lonardi, S., & Chiu, B. (2003) · doi:10.1145/882085.882086 · Cited by 642
Towards parameter-free data mining
Keogh, E., Lonardi, S., & Ratanamahatana, C. A. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014077 · Cited by 567
The reduced nearest neighbor rule (Corresp.)
Gates, G. (1972) · IEEE Transactions on Information Theory · doi:10.1109/tit.1972.1054809 · Cited by 566
k-Nearest Neighbour Classifiers
Cunningham, P., & Delany, S. J. (2007) · Research Repository UCD (University College Dublin) · Cited by 542
Self-citationAdvances in Instance Selection for Instance-Based Learning Algorithms
Brighton, H., & Mellish, C. (2002) · Data Mining and Knowledge Discovery · doi:10.1023/a:1014043630878 · Cited by 532
Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality
Indyk, P., & Motwani, R. (1998) · Cited by 486
An Experiment with the Edited Nearest-Neighbor Rule
(1976) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1976.4309523 · Cited by 470
11th International Joint Conference on Artificial Intelligence
(1988) · Cognitive Science · doi:10.1207/s15516709cog1202_7 · Cited by 467
Multidimensional Binary Search Trees in Database Applications
Bentley, J. L. (1979) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1979.234200 · Cited by 410
Random projection trees and low dimensional manifolds
Dasgupta, S., & Freund, Y. (2008) · Proceedings of the fortieth annual ACM symposium on Theory of computing · doi:10.1145/1374376.1374452 · Cited by 375
An Investigation of Practical Approximate Nearest Neighbor Algorithms
Liu, T., Moore, A., Yang, K., & Gray, A. (2004) · Cited by 375
Two algorithms for nearest-neighbor search in high dimensions
Kleinberg, J. M. (1997) · Proceedings of the twenty-ninth annual ACM symposium on Theory of computing - STOC '97 · doi:10.1145/258533.258653 · Cited by 348
An algorithm for a selective nearest neighbor decision rule (Corresp.)
Ritter, G., Woodruff, H., Lowry, S., & Isenhour, T. (1975) · IEEE Transactions on Information Theory · doi:10.1109/tit.1975.1055464 · Cited by 323
Overfitting in making comparisons between variable selection methods
ReunanenJuha (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944978 · Cited by 205
SFA: a symbolic fourier approximation and index for similarity search in high dimensional datasets
Schäfer, P., & Högqvist, M. (2012) · International Conference on Extending Database Technology (EDBT) · doi:10.1145/2247596.2247656 · Cited by 195
Wilson, D. R., & Martinez, T. R. (1997) · Cited by 185
Fast Nearest Neighbor Condensation for Large Data Sets Classification
Angiulli, F. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.190645 · Cited by 177
Feature subset selection in text-learning
Mladenić, D. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026677 · Cited by 171
Survey on distance metric learning and dimensionality reduction in data mining
Wang, F., & Sun, J. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0356-z · Cited by 168
Machine Learning Techniques for Multimedia
Cord, M., & Cunningham, P. (2008) · Cognitive technologies · doi:10.1007/978-3-540-75171-7 · Cited by 152
Self-citationAddressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection
Brodley, C. E. (1993) · Elsevier eBooks · doi:10.1016/b978-1-55860-307-3.50009-5 · Cited by 151
What Is a Good Nearest Neighbors Algorithm for Finding Similar Patches in Images?
Kumar, N., Zhang, L., & Nayar, S. K. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88688-4_27 · Cited by 136
PCA vs. ICA: A Comparison on the FERET Data Set.
Baek, K., Draper, B. A., Beveridge, J. R., & She, K. (2002) · Cited by 122
Remembering to Forget: The Amnesic Effect of Daydreaming
Delaney, P. F., Sahakyan, L., Kelley, C. M., & Zimmerman, C. A. (2010) · Psychological Science · doi:10.1177/0956797610374739 · Cited by 106
An Analysis of Case-Base Editing in a Spam Filtering System
Delany, S. J., & Cunningham, P. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-28631-8_11 · Cited by 79
Self-citationFeature Selection Using Improved Mutual Information for Text Classification
Novovičová, J., Malík, A., & Pudil, P. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-27868-9_111 · Cited by 36
Scoring Performance on the Y-Balance Test
Mahato, V., Johnston, W., & Cunningham, P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29249-2_19 · Cited by 7
Self-citation
Cited by
Huang, S.-C., Pareek, A., Jensen, M., Lungren, M. P., Yeung, S., & Chaudhari, A. S. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00811-0 · Cited by 479
Stroke Risk Prediction with Machine Learning Techniques
Dritsas, E., & Trigka, M. (2022) · Sensors · doi:10.3390/s22134670 · Cited by 286
Machine Learning Applications in Agriculture: Current Trends, Challenges, and Future Perspectives
Araújo, S. O., Peres, R. S., Ramalho, J. C., Lidon, F., & Barata, J. (2023) · Agronomy · doi:10.3390/agronomy13122976 · Cited by 259
Performance Prediction of Cement Stabilized Soil Incorporating Solid Waste and Propylene Fiber
Zhang, G., Ding, Z., Wang, Y., Fu, G., Wang, Y., Xie, C., Zhang, Y., Zhao, X., Lu, X., & Wang, X. (2022) · Materials · doi:10.3390/ma15124250 · Cited by 24
On Predicting Transitions to Compliant Surfaces in Human Gait via Neural and Kinematic Signals
Angelidou, C., & Artemiadis, P. (2023) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2023.3272355 · Cited by 19
Metadata
- Authors
- Cunningham, P., & Delany, S. J.
- Venue
- ACM Computing Surveys
- Published
- 2021-07-13
- Volume
- 54
- Issue
- 6
- Pages
- 1-25
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3015461837
- Cited by
- 942
- References
- 50
- FWCI
- 128.34
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.