The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Effects of Distance Measure Choice on K-Nearest Neighbor Classifier Performance: A Review
Open accessAbstract
The K-nearest neighbor (KNN) classifier is one of the simplest and most common classifiers, yet its performance competes with the most complex classifiers in the literature. The core of this classifier depends mainly on measuring the distance or similarity between the tested examples and the training examples. This raises a major question about which distance measures to be used for the KNN classifier among a large number of distance and similarity measures available? This review attempts to answer this question through evaluating the performance (measured by accuracy, precision, and recall) of the KNN using a large number of distance measures, tested on a number of real-world data sets, with and without adding different levels of noise. The experimental results show that the performance of KNN classifier depends significantly on the distance used, and the results showed large gaps between the performances of different distances. We found that a recently proposed nonconvex distance performed the best when applied on most data sets comparing with the other tested distances. In addition, the performance of the KNN with this top performing distance degraded only ∼20% while the noise level reaches 90%, this is true for most of the distances used as well. This means that the KNN classifier using any of the top 10 distances tolerates noise to a certain degree. Moreover, the results show that some distances are less affected by the added noise comparing with other distances.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354
Measures of the Amount of Ecologic Association Between Species
Dice, L. R. (1945) · Ecology · doi:10.2307/1932409 · Cited by 11,874
Encyclopedia of Analytical Chemistry: Applications, Theory and Instrumentation
(2006) · Wiley eBooks · doi:10.1002/9780470027318 · Cited by 7,403
Top 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667
Derrac, J., García, S., Molina, D., & Herrera, F. (2011) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2011.02.002 · Cited by 5,512
A detailed analysis of the KDD CUP 99 data set
Tavallaee, M., Bagheri, E., Lu, W., & Ghorbani, A. A. (2009) · 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications · doi:10.1109/cisda.2009.5356528 · Cited by 5,396
Error Detecting and Error Correcting Codes
Hamming, R. W. (1950) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1950.tb00463.x · Cited by 5,377
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_16 · Cited by 2,915
Étude comparative de la distribution florale dans une portion des Alpes et du Jura
Jaccard, P. (1901) · E-Periodica · doi:10.5169/seals-266450 · Cited by 2,353
An invariant form for the prior probability in estimation problems
Jeffreys, H. (1946) · Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences · doi:10.1098/rspa.1946.0056 · Cited by 2,323
Willett, P., Barnard, J. M., & Downs, G. M. (1998) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci9800211 · Cited by 1,765
The condensed nearest neighbor rule (Corresp.)
Hart, P. (1968) · IEEE Transactions on Information Theory · doi:10.1109/tit.1968.1054155 · Cited by 1,733
Discriminatory analysis: Nonparametric discrimination: Consistency properties: (471672008-001)
Fix, E., & Hodges, J. L. (1951) · PsycEXTRA Dataset · doi:10.1037/e471672008-001 · Cited by 1,696
Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions
Cha, S.-H. (2007) · Cited by 1,644
Data Clustering: Theory, Algorithms, and Applications
Gan, G., Ma, C., & Wu, J. (2007) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898718348 · Cited by 1,562
Data Preprocessing in Data Mining
García, S., Luengo, J., & Herrera, F. (2015) · Intelligent Systems Reference Library · doi:10.1007/978-3-319-10247-4 · Cited by 1,368
Reduction Techniques for Instance-Based Learning Algorithms
Wilson, D. R., & Martinez, T. R. (2000) · Machine Learning · doi:10.1023/a:1007626913721 · Cited by 1,182
Class Noise vs. Attribute Noise: A Quantitative Study
Zhu, X., & Wu, X. (2004) · Artificial Intelligence Review · doi:10.1007/s10462-004-0751-8 · Cited by 893
The thirteen books of Euclid's Elements
Euclid (1956) · CERN Document Server (European Organization for Nuclear Research) · Cited by 878
Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen.
Hellinger, E. (1909) · Journal für die reine und angewandte Mathematik · doi:10.1515/crll.1909.136.210 · Cited by 804
Wettschereck, D., Aha, D. W., & Mohri, T. (1997) · Artificial Intelligence Review · doi:10.1023/a:1006593614256 · Cited by 717
A Study of Summer Foliage Insect Communities in the Great Smoky Mountains
Whittaker, R. H. (1952) · Ecological Monographs · doi:10.2307/1948527 · Cited by 616
Efficient kNN classification algorithm for big data
Deng, Z., Zhu, X., Cheng, D., Zong, M., & Zhang, S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.08.112 · Cited by 603
The reduced nearest neighbor rule (Corresp.)
Gates, G. (1972) · IEEE Transactions on Information Theory · doi:10.1109/tit.1972.1054809 · Cited by 566
The distance function effect on k-nearest neighbor classification for medical datasets
Hu, L.-Y., Huang, M.-W., Ke, S.-W., & Tsai, C.-F. (2016) · SpringerPlus · doi:10.1186/s40064-016-2941-7 · Cited by 455
Pearson, K. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_2 · Cited by 454
Survey of Nearest Neighbor Techniques
Bhatia, N., & Vandana (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1007.0085 · Cited by 447
A study of the effect of different types of noise on the precision of supervised learning techniques
Nettleton, D. F., Orriols-Puig, A., & Fornells, A. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-010-9156-z · Cited by 446
Error Detecting and Error Correcting Codes (1950)
Hamming, R. W. (2021) · Ideas That Created the Future · doi:10.7551/mitpress/12274.003.0015 · Cited by 429
An Agglomerative Method for Classification of Plant Communities
Orloci, L. (1967) · The Journal of Ecology · doi:10.2307/2257725 · Cited by 419
A Comparison Study on Similarity and Dissimilarity Measures in Clustering Continuous Data
Shirkhorshidi, A. S., Aghabozorgi, S., & Wah, T. Y. (2015) · PLOS ONE · doi:10.1371/journal.pone.0144059 · Cited by 405
Computer Programs for Hierarchical Polythetic Classification ("Similarity Analyses")
Lance, G. N., & Williams, W. T. (1966) · The Computer Journal · doi:10.1093/comjnl/9.1.60 · Cited by 402
Mixed-Data Classificatory Programs I - Agglomerative Systems.
Lance, G. N., & Williams, W. T. (1967) · Australian Computer Journal · Cited by 338
Data Analysis in Management with SPSS Software
Verma, J. P. (2013) · Springer eBooks · doi:10.1007/978-81-322-0786-3 · Cited by 333
Some inequalities for information divergence and related measures of discrimination
Topsoe, F. (2000) · IEEE Transactions on Information Theory · doi:10.1109/18.850703 · Cited by 325
kNN-IS: An Iterative Spark-based design of the k-Nearest Neighbors classifier for big data
Maillo, J., Ramírez, S., Triguero, I., & Herrera, F. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.06.012 · Cited by 316
Approximate nearest neighbor queries in fixed dimensions
Arya, S., & Mount, D. M. (1993) · doi:10.5555/313559.313768 · Cited by 303
Perceptual Metrics for Image Database Navigation
Rubner, Y., & Tomasi, C. (2001) · Springer eBooks · doi:10.1007/978-1-4757-3343-3 · Cited by 296
An Empirical Study of Distance Metrics for k-Nearest Neighbor Algorithm
Chomboon, K., Chujai, P., Teerarassammee, P., Kerdprasop, K., & Kerdprasop, N. (2015) · The Proceedings of the 2nd International Conference on Industrial Application Engineering 2015 · doi:10.12792/iciae2015.051 · Cited by 199
Solving the Problem of the K Parameter in the KNN Classifier Using an Ensemble Learning Approach
Hassanat, A. B. A., Abbadi, M. A., Tarawneh, A. S., & Alhasanat, A. A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.0919 · Cited by 168
Self-citationQuery dependent ranking using K-nearest neighbor
Geng, X., Liu, T.-Y., Qin, T., Arnold, A., Li, H., & Shum, H.-Y. (2008) · Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1390334.1390356 · Cited by 165
Gallego, A., Calvo-Zaragoza, J., Valero-Mas, J. J., & Rico-Juan, J. R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.09.038 · Cited by 116
New Developments in Generalized Information Measures
Taneja, I. J. (1995) · Advances in imaging and electron physics · doi:10.1016/s1076-5670(08)70106-x · Cited by 110
Sáez, J. A., Galar, M., Luengo, J., & Herrera, F. (2013) · Information Sciences · doi:10.1016/j.ins.2013.06.002 · Cited by 100
Wettschereck, D., Aha, D. W., & Mohri, T. (1997) · Springer eBooks · doi:10.1007/978-94-017-2053-3_11 · Cited by 97
LazyLSH: Approximate Nearest Neighbor Search for Multiple Distance Functions with a Single Index
Zheng, Y., Guo, Q., Tung, A. K. H., & Wu, S. (2016) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2882903.2882930 · Cited by 82
Software Similarity and Classification
Cesare, S., & Xiang, Y. (2012) · SpringerBriefs in computer science · doi:10.1007/978-1-4471-2909-7 · Cited by 67
An Empirical Comparison of Dissimilarity Measures for Time Series Classification
Giusti, R., & Batista, G. E. A. P. A. (2013) · Brazilian Conference on Intelligent Systems · doi:10.1109/bracis.2013.22 · Cited by 66
Efficient tree classifiers for large scale datasets
Wang, F., Wang, Q., Nie, F., Yu, W., & Wang, R. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.12.061 · Cited by 47
Todeschini, R., Ballabio, D., Consonni, V., & Grisoni, F. (2016) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2016.06.013 · Cited by 34
Distances and Other Dissimilarity Measures in Chemometrics
Todeschini, R., Ballabio, D., & Consonni, V. (2015) · Wiley eBooks · doi:10.1002/9780470027318.a9438 · Cited by 30
Furthest-Pair-Based Decision Trees: Experimental Results on Big Data Classification
Hassanat, A. B. A. (2018) · Information · doi:10.3390/info9110284 · Cited by 26
Self-citationExperiments of Distance Measurements in a Foliage Plant Retrieval SystemPreprint
Kadir, A., Nugroho, L. E., Susanto, A., et al. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.3584 · Cited by 11
Cited by
Choosing Mutation and Crossover Ratios for Genetic Algorithms—A Review with a New Dynamic Approach
Hassanat, A., Almohammadi, K., Alkafaween, E., Abunawas, E., Hammouri, A., & Prasath, V. B. S. (2019) · Information · doi:10.3390/info10120390 · Cited by 668
Self-citationBotchkarev, A. (2019) · Interdisciplinary Journal of Information, Knowledge, and Management · doi:10.28945/4184 · Cited by 616
Botchkarev, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03006 · Cited by 387
Wang, B., Li, Y., Zhou, M., Han, Y., Zhang, M., Gao, Z., Liu, Z., Chen, P., Du, W., Zhang, X., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-36017-x · Cited by 259
Machine Learning in Pharmacometrics: Opportunities and Challenges
McComb, M., Bies, R., & Ramanathan, M. (2022) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.14801 · Cited by 128
Metadata
- Authors
- Abu Alfeilat, H. A., Hassanat, A. B. A., Lasassmeh, O., Tarawneh, A. S., Alhasanat, M. B., Eyal Salman, H. S., & Prasath, V. B. S.
- Venue
- Big Data
- Publisher
- SAGE Publications
- Published
- 2019-12-01
- Volume
- 7
- Issue
- 4
- Pages
- 221-248
- ISSN
- 2167-6461 (ISSN-L)
- 2167-647X
- OpenAlex
- W3098996042
- Cited by
- 667
- References
- 56
- FWCI
- 26.82
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.