The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
A comparative study of efficient initialization methods for the k-means clustering algorithm
Open accessAbstract
K-means is undoubtedly the most widely used partitional clustering algorithm. Unfortunately, due to its gradient descent nature, this algorithm is highly sensitive to the initial placement of the cluster centers. Numerous initialization methods have been proposed to address this problem. In this paper, we first present an overview of these methods with an emphasis on their computational efficiency. We then compare eight commonly used linear time complexity initialization methods on a large and diverse collection of data sets using various performance criteria. Finally, we analyze the experimental results using non-parametric statistical tests and provide recommendations for practitioners. We demonstrate that popular initialization methods often perform poorly and that there are in fact strong alternatives to these methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,387
(1967) · Cited by 23,719
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Least squares quantization in PCM
Lloyd, S. (1982) · IEEE Transactions on Information Theory · doi:10.1109/tit.1982.1056489 · Cited by 15,836
Algorithm AS 136: A K-Means Clustering Algorithm
Hartigan, J. A., & Wong, M. A. (1979) · Applied Statistics · doi:10.2307/2346830 · Cited by 14,523
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Jain, A. K., Murty, M. N., & Flynn, P. J. (1999) · ACM Computing Surveys · doi:10.1145/331499.331504 · Cited by 13,199
Finding groups in data: an introduction to cluster analysis
(1991) · Choice Reviews Online · doi:10.5860/choice.28-4558 · Cited by 10,474
Data clustering: 50 years beyond K-means
Jain, A. K. (2010) · Pattern Recognition Letters · doi:10.1016/j.patrec.2009.09.011 · Cited by 9,368
Hubert, L., & Arabie, P. (1985) · Journal of Classification · doi:10.1007/bf01908075 · Cited by 8,219
(2007) · doi:10.5555/1283383.1283494 · Cited by 7,364
An Algorithm for Vector Quantizer Design
Linde, Y., Buzo, A., & Gray, R. (1980) · IEEE Transactions on Communications · doi:10.1109/tcom.1980.1094577 · Cited by 7,201
The use of ranks to avoid the assumption of normality implicit in the analysis of variance
Friedman, M. (1937) · Journal of the American Statistical Association · doi:10.1080/01621459.1937.10503522 · Cited by 7,005
Fast and robust fixed-point algorithms for independent component analysis
Hyvarinen, A. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.761722 · Cited by 6,246
Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator
Matsumoto, M., & Nishimura, T. (1998) · ACM Transactions on Modeling and Computer Simulation · doi:10.1145/272991.272995 · Cited by 5,780
An efficient k-means clustering algorithm: analysis and implementation
Kanungo, T., Mount, D. M., Netanyahu, N. S., Piatko, C. D., Silverman, R., & Wu, A. Y. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017616 · Cited by 5,660
LOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335388 · Cited by 5,207
Cluster Analysis for Applications
(1973) · Elsevier eBooks · doi:10.1016/c2013-0-06161-0 · Cited by 5,080
Applied Nonparametric Statistics
Wozniak, P. J., & Daniel, W. W. (1991) · Technometrics · doi:10.2307/1268795 · Cited by 4,987
The global k-means clustering algorithm
Likas, A., Vlassis, N., & J. Verbeek, J. (2003) · Pattern Recognition · doi:10.1016/s0031-3203(02)00060-2 · Cited by 3,733
Finding Groups in Data: An Introduction to Cluster Analysis.
Sarle, W. S., Kaufman, L., & Rousseeuw, P. J. (1991) · Journal of the American Statistical Association · doi:10.2307/2290430 · Cited by 3,539
Pattern Recognition Principles
(2009) · Classification Methods for Remotely Sensed Data, Second Edition · doi:10.1201/9781420090741.ch2 · Cited by 3,188
Cluster analysis of multivariate data : efficiency versus interpretability of classifications
Forgy, E. W. (1965) · Biometrics · Cited by 2,093
Clustering to minimize the maximum intercluster distance
Gonzalez, T. F. (1985) · Theoretical Computer Science · doi:10.1016/0304-3975(85)90224-5 · Cited by 1,717
A General Theory of Classificatory Sorting Strategies: 1. Hierarchical Systems
Lance, G. N., & Williams, W. T. (1967) · The Computer Journal · doi:10.1093/comjnl/9.4.373 · Cited by 1,696
García, S., & Herrera, F. (2008) · Institutional Repository of the University of Granada (University of Granada) · Cited by 1,619
Comparing clusterings—an information based distance
Meilă, M. (2007) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2006.11.013 · Cited by 1,546
K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality
Selim, S. Z., & Ismail, M. A. (1984) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1984.4767478 · Cited by 1,212
An Examination of the Effect of Six Types of Error Perturbation on Fifteen Clustering Algorithms
Milligan, G. W. (1980) · Psychometrika · doi:10.1007/bf02293907 · Cited by 1,162
Approximations of the critical region of the fbietkan statistic
Iman, R. L., & Davenport, J. M. (1980) · Communications in Statistics - Theory and Methods · doi:10.1080/03610928008827904 · Cited by 1,125
Statistical Comparisons of Classifiers over Multiple Data Sets
DemšarJanez (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248548 · Cited by 1,074
Pattern Recognition Principles
Kavzoglu, T., Tso, B., & Mather, P. M. (2024) · Classification Methods for Remotely Sensed Data · doi:10.1201/9781003439172-2 · Cited by 1,053
Refining Initial Points for K-Means Clustering
Bradley, P. S., & Fayyad, U. M. (1998) · Cited by 1,031
A clustering technique for summarizing multivariate data
Ball, G. H., & Hall, D. J. (1967) · Behavioral Science · doi:10.1002/bs.3830120210 · Cited by 912
A study of standardization of variables in cluster analysis
Milligan, G. W., & Cooper, M. C. (1988) · Journal of Classification · doi:10.1007/bf01897163 · Cited by 877
NP-hardness of Euclidean sum-of-squares clustering
Aloise, D., Deshpande, A., Hansen, P., & Popat, P. (2009) · Machine Learning · doi:10.1007/s10994-009-5103-0 · Cited by 867
An empirical comparison of four initialization methods for the K-Means algorithm
Peña, J. M., Lozano, J. A., & Larrañaga, P. (1999) · Pattern Recognition Letters · doi:10.1016/s0167-8655(99)00069-0 · Cited by 866
García, S., Fernández, A., Luengo, J., & Herrera, F. (2009) · Soft Computing · doi:10.1007/s00500-008-0392-y · Cited by 682
Parallel Spectral Clustering in Distributed Systems
Chen, W.-Y., Song, Y., Bai, H., Lin, C.-J., & Chang, E. Y. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.88 · Cited by 603
An improvement of the minimum distortion encoding algorithm for vector quantization
Bei, C.-D., & Gray, R. M. (1985) · IEEE Transactions on Communications · doi:10.1109/tcom.1985.1096214 · Cited by 411
Convergence Properties of the K-Means Algorithms
Bottou, L., & Bengio, Y. (1994) · Cited by 398
A new initialization technique for generalized Lloyd iteration
Katsavounidis, I. X., Kuo, C. ‐. J., & Zhang, Z. (1994) · IEEE Signal Processing Letters · doi:10.1109/97.329844 · Cited by 372
Relative clustering validity criteria: A comparative overview
Vendramin, L., Campello, R. J. G. B., & Hruschka, E. R. (2010) · Statistical Analysis and Data Mining: The ASA Data Science Journal · doi:10.1002/sam.10080 · Cited by 325
The Planar k-Means Problem is NP-Hard
Mahajan, M., Nimbhorkar, P., & Varadarajan, K. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-00202-1_24 · Cited by 308
IBM SPSS Statistics 19 Statistical Procedures Companion
Norušis, M. J. (2011) · Cited by 294
A self-organizing network for hyperellipsoidal clustering (HEC)
Mao, J., & Jain, A. K. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.478389 · Cited by 281
The planark-means problem is NP-hard
Mahajan, M., Nimbhorkar, P., & Varadarajan, K. (2012) · Theoretical Computer Science · doi:10.1016/j.tcs.2010.05.034 · Cited by 277
A general theory of classificatory sorting strategies: II. Clustering systems
Lance, G. N. (1967) · The Computer Journal · doi:10.1093/comjnl/10.3.271 · Cited by 269
A method for initialising the K-means clustering algorithm using kd-trees
Redmond, S. J., & Heneghan, C. (2007) · Pattern Recognition Letters · doi:10.1016/j.patrec.2007.01.001 · Cited by 261
Adapting the right measures for K-means clustering
Wu, J., Xiong, H., & Chen, J. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557115 · Cited by 255
Performance criteria for graph clustering and Markov cluster experiments
van Dongen, S. (2000) · Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands · Cited by 216
Multidimensional group analysis
Jancey, R. (1966) · Australian Journal of Botany · doi:10.1071/bt9660127 · Cited by 193
Hamerly, G. (2010) · Proceedings of the 2010 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972801.12 · Cited by 190
Fuzzy Sets and Decisionmaking Approaches in Vowel and Speaker Recognition
(1977) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1977.4309789 · Cited by 186
Improving the performance of k-means for color quantization
Celebi, M. E. (2011) · Image and Vision Computing · doi:10.1016/j.imavis.2010.10.002 · Cited by 179
Self-citationLuengo, J., García, S., & Herrera, F. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.11.041 · Cited by 173
Improvements of General Multiple Test Procedures for Redundant Systems of Hypotheses
Bergmann, B., & Hommel, G. (1988) · Medizinische Informatik und Statistik · doi:10.1007/978-3-642-52307-6_8 · Cited by 162
Simulating Data to Study Performance of Finite Mixture Modeling and Clustering Algorithms
Maitra, R., & Melnykov, V. (2010) · Journal of Computational and Graphical Statistics · doi:10.1198/jcgs.2009.08054 · Cited by 159
In search of deterministic methods for initializing K-means and Gaussian mixture clustering
Su, T., & Dy, J. (2007) · Intelligent Data Analysis · doi:10.3233/ida-2007-11402 · Cited by 133
An initialization method for the K -Means algorithm using neighborhood model
Cao, F., Liang, J., & Jiang, G. (2009) · Computers & Mathematics with Applications · doi:10.1016/j.camwa.2009.04.017 · Cited by 133
Initialization of cluster refinement algorithms: a review and comparative study
He, J., Lan, M., Tan, C.-L., Sung, S.-Y., & Low, H.-B. (2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1379917 · Cited by 108
Efficient disk-based K-means clustering for relational databases
ORDÓÑEZ, C. R., & Omiecinski, E. R. (2004) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2004.25 · Cited by 97
A comparison of several vector quantization codebook generation approaches
Huang, C.-C., & Harris, R. W. (1993) · IEEE Transactions on Image Processing · doi:10.1109/83.210871 · Cited by 91
New methods for the initialisation of clusters
Al-Daoud, M. B., & Roberts, S. (1996) · Pattern Recognition Letters · doi:10.1016/0167-8655(95)00119-0 · Cited by 82
Fast encoding algorithm for VQ-based image coding
Huang, S.-H., & Chen, S. (1990) · Electronics Letters · doi:10.1049/el:19901037 · Cited by 80
Improved step size adaptation for the MO-CMA-ES
Voß, T., Hansen, N., & Igel, C. (2010) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1830483.1830573 · Cited by 77
External validation measures for K-means clustering: A data distribution perspective
Wu, J., Chen, J., Xiong, H., & Xie, M. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.06.093 · Cited by 74
Robust partitional clustering by outlier and density insensitive seeding
Hasan, M. A., Chaoji, V., Salem, S., & Zaki, M. J. (2009) · Pattern Recognition Letters · doi:10.1016/j.patrec.2009.04.013 · Cited by 70
'1+1>2': merging distance and density based clustering
Dash, M., Liu, H., & Xu, X. (2001) · Database Systems for Advanced Applications · doi:10.1109/dasfaa.2001.916361 · Cited by 59
Hommel, G., & Bernhard, G. (1993) · Contributions to statistics · doi:10.1007/978-3-642-99766-2_10 · Cited by 58
Robust clustering by pruning outliers
Zhang, J., & Leung, Y.-W. (2003) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2003.816993 · Cited by 47
Bandwidth Adaptive Hardware Architecture of K-Means Clustering for Video Analysis
Chen, T.-W., & Chien, S. (2009) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2009.2017543 · Cited by 45
A Divisive Initialisation Method for Clustering Algorithms
Pizzuti, C., Talia, D., & Vonella, G. (1999) · Lecture notes in computer science · doi:10.1007/978-3-540-48247-5_62 · Cited by 10
Bangdiwala, S. I. (2017) · International Journal of Injury Control and Safety Promotion · doi:10.1080/17457300.2016.1270401 · Cited by 10
Cited by
Ikotun, A. M., Ezugwu, A. E., Abualigah, L., Abuhaija, B., & Heming, J. (2023) · Information Sciences · doi:10.1016/j.ins.2022.11.139 · Cited by 2,272
Syakur, M. A., Khotimah, B. K., Rochman, E. M. S., & Satoto, B. D. (2018) · IOP Conference Series: Materials Science and Engineering · doi:10.1088/1757-899x/336/1/012017 · Cited by 1,181
How much can k-means be improved by using better initialization and repeats?
Fränti, P., & Sieranoja, S. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.04.014 · Cited by 451
Rashidi, H. H., Tran, N. K., Betts, E. V., Howell, L. P., & Green, R. (2019) · Academic Pathology · doi:10.1177/2374289519873088 · Cited by 371
Continental-scale animal tracking reveals functional movement classes across marine taxa
Brodie, S., Lédée, E. J. I., Heupel, M. R., Babcock, R. C., Campbell, H. A., Gledhill, D. C., Hoenner, X., Huveneers, C., Jaine, F. R. A., Simpfendorfer, C. A., et al. (2018) · Scientific Reports · doi:10.1038/s41598-018-21988-5 · Cited by 346
Metadata
- Authors
- Celebi, M. E., Kingravi, H. A., & Vela, P. A.
- Venue
- Expert Systems with Applications
- Publisher
- Elsevier BV
- Published
- 2013-01
- Volume
- 40
- Issue
- 1
- Pages
- 200-210
- ISSN
- 0957-4174 (ISSN-L)
- OpenAlex
- W2059515884
- Cited by
- 1,214
- References
- 74
- FWCI
- 34.20
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.