The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Relative clustering validity criteria: A comparative overview
Abstract
Many different relative clustering validity criteria exist that are very useful in practice as quantitative measures for evaluating the quality of data partitions, and new criteria have still been proposed from time to time. These criteria are endowed with particular features that may make each of them able to outperform others in specific classes of problems. In addition, they may have completely different computational requirements. Then, it is a hard task for the user to choose a specific criterion when he or she faces such a variety of possibilities. For this reason, a relevant issue within the field of clustering analysis consists of comparing the performances of existing validity criteria and, eventually, that of a new criterion to be proposed. In spite of this, the comparison paradigm traditionally adopted in the literature is subject to some conceptual limitations. The present paper describes an alternative, possibly complementary methodology for comparing clustering validity criteria and uses it to make an extensive comparison of the performances of 40 criteria over a collection of 962,928 partitions derived from five well‐known clustering algorithms and 1080 different data sets of a given class of interest. A detailed review of the relative criteria under investigation is also provided that includes an original comparative asymptotic analysis of their computational complexities. This work is intended to be a complement of the classic study reported in 1985 by Milligan and Cooper as well as a thorough extension of a preliminary paper by the authors themselves. Copyright © 2010 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 3: 209‐235, 2010
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Silhouettes: A graphical aid to the interpretation and validation of cluster analysis
Rousseeuw, P. J. (1987) · Journal of Computational and Applied Mathematics · doi:10.1016/0377-0427(87)90125-7 · Cited by 21,392
Jain, A. K., Murty, M. N., & Flynn, P. J. (1999) · ACM Computing Surveys · doi:10.1145/331499.331504 · Cited by 13,199
Davies, D. L., & Bouldin, D. W. (1979) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1979.4766909 · Cited by 9,145
Hubert, L., & Arabie, P. (1985) · Journal of Classification · doi:10.1007/bf01908075 · Cited by 8,219
Finding Groups in Data: An Introduction to Cluster Analysis
Kaufman, L., & Rousseeuw, P. J. (1990) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316801 · Cited by 7,344
Objective Criteria for the Evaluation of Clustering Methods
Rand, W. M. (1971) · Journal of the American Statistical Association · doi:10.1080/01621459.1971.10482356 · Cited by 6,566
Algorithms for clustering data
Flury, B. (1989) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(89)90033-5 · Cited by 6,432
Survey of Clustering Algorithms
Xu, R., & WunschII, D. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.845141 · Cited by 6,197
Top 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667
Maximum likelihood estimation from incomplete data via the EM algorithm
Dempster, A. P. (1977) · Journal of the Royal Statistical Society Series A (Statistics in Society) · Cited by 4,543
An Examination of Procedures for Determining the Number of Clusters in a Data Set
Milligan, G. W., & Cooper, M. C. (1985) · Psychometrika · doi:10.1007/bf02294245 · Cited by 3,868
On Clustering Validation Techniques
Halkidi, M., Batistakis, Y., & Vazirgiannis, M. (2001) · Journal of Intelligent Information Systems · doi:10.1023/a:1012801612483 · Cited by 2,646
Well-Separated Clusters and Optimal Fuzzy Partitions
Dunn†, J. C. (1974) · Journal of Cybernetics · doi:10.1080/01969727408546059 · Cited by 2,510
A Method for Comparing Two Hierarchical Clusterings
Fowlkes, E. B., & Mallows, C. L. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10478008 · Cited by 1,614
Hoeffding, W., & Kendall, M. G. (1957) · Econometrica · doi:10.2307/1907752 · Cited by 1,569
Data Clustering: Theory, Algorithms, and Applications
Gan, G., Ma, C., & Wu, J. (2007) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898718348 · Cited by 1,562
Sibson, R., & Hartigan, J. A. (1976) · Applied Statistics · doi:10.2307/2346526 · Cited by 1,435
Performance evaluation of some clustering algorithms and validity indices
Maulik, U., & Bandyopadhyay, S. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1114856 · Cited by 1,396
Cluster analysis for gene expression data: a survey
Jiang, D., Tang, C., & Zhang, A. (2004) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2004.68 · Cited by 1,235
Some new indexes of cluster validity
Bezdek, J. C., & Pal, N. R. (1998) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/3477.678624 · Cited by 1,132
ISODATA, A NOVEL METHOD OF DATA ANALYSIS AND PATTERN CLASSIFICATION
Ball, G. H., & Hall, D. J. (1965) · Defense Technical Information Center (DTIC) · Cited by 919
A Survey of Evolutionary Algorithms for Clustering
Hruschka, E. R., Campello, R. J. G. B., Freitas, A. A., & de Carvalho, A. C. P. L. F. (2009) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2008.2007252 · Cited by 759
Self-citationZiegel, E. R. (2002) · Technometrics · doi:10.1198/tech.2002.s94 · Cited by 715
Measures of Association for Cross Classifications*
Goodman, L. A., & Kruskal, W. H. (1954) · Journal of the American Statistical Association · doi:10.1080/01621459.1954.10501231 · Cited by 678
On Some Invariant Criteria for Grouping Data
Friedman, H. P., & Rubin, J. (1967) · Journal of the American Statistical Association · doi:10.1080/01621459.1967.10500923 · Cited by 613
Numerical methods for fuzzy clustering
Ruspini, E. H. (1970) · Information Sciences · doi:10.1016/s0020-0255(70)80056-1 · Cited by 561
A Study of the Comparability of External Criteria for Hierarchical Cluster Analysis
Milligan, G. W., & Cooper, M. C. (1986) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2104_5 · Cited by 482
Clustering validity checking methods: part II
Halkidi, M., Batistakis, Y., & Vazirgiannis, M. (2002) · ACM SIGMOD Record · doi:10.1145/601858.601862 · Cited by 466
Clustering Methods Based on Likelihood Ratio Criteria
Scott, A. J., & Symons, M. J. (1971) · Biometrics · doi:10.2307/2529003 · Cited by 464
A Monte Carlo Study of Thirty Internal Criterion Measures for Cluster Analysis
Milligan, G. W. (1981) · Psychometrika · doi:10.1007/bf02293899 · Cited by 407
A general statistical framework for assessing categorical clustering in free recall.
Hubert, L. J., & Levin, J. R. (1976) · Psychological Bulletin · doi:10.1037//0033-2909.83.6.1072 · Cited by 398
Measuring the Power of Hierarchical Cluster Analysis
Baker, F. B., & Hubert, L. J. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10480256 · Cited by 272
How many clusters are best? - An experiment
Dubes, R. C. (1987) · Pattern Recognition · doi:10.1016/0031-3203(87)90034-3 · Cited by 226
Practical Problems in a Method of Cluster Analysis
Marriott, F. H. C. (1971) · Biometrics · doi:10.2307/2528592 · Cited by 222
A Method for Comparing Two Hierarchical Clusterings: Comment
Wallace, D. L. (1983) · Journal of the American Statistical Association · doi:10.2307/2288118 · Cited by 169
Methods of Comparing Classifications
Rohlf, F. J. (1974) · Annual Review of Ecology and Systematics · doi:10.1146/annurev.es.05.110174.000533 · Cited by 165
Evolving clusters in gene-expression data
Hruschka, E. R., Campello, R. J. G. B., & de Castro, L. N. (2005) · Information Sciences · doi:10.1016/j.ins.2005.07.015 · Cited by 157
Self-citationOn the Comparison of Relative Clustering Validity Criteria
Vendramin, L., Campello, R. J. G. B., & Hruschka, E. R. (2009) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972795.63 · Cited by 131
Self-citationArnold, S. J. (1979) · Journal of Marketing Research · doi:10.1177/002224377901600411 · Cited by 64
(1999) · CRC Press eBooks · doi:10.1201/9781420049268.ch4 · Cited by 53
C129. An index of separateness of clusters and a permutation test for its statistical significance
Good, I. J. (1982) · Journal of Statistical Computation and Simulation · doi:10.1080/00949658208810568 · Cited by 44
Comparing Two Partitions: Some Proposals and Experiments
Saporta, G., & Youness, G. (2002) · Physica-Verlag eBooks · doi:10.1007/978-3-642-57489-4_33 · Cited by 37
A Stopping Rule for Partitioning Dendrograms
Hill, R. S. (1980) · Botanical Gazette · doi:10.1086/337163 · Cited by 33
Cited by
Kriegel, H., Kröger, P., Sander, J., & Zimek, A. (2011) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.30 · Cited by 900
Metadata
- Authors
- Vendramin, L., Campello, R. J. G. B., & Hruschka, E. R.
- Venue
- Statistical Analysis and Data Mining: The ASA Data Science Journal
- Publisher
- Wiley
- Published
- 2010-07-20
- Volume
- 3
- Issue
- 4
- Pages
- 209-235
- ISSN
- 1932-1864 (ISSN-L)
- 1932-1872
- OpenAlex
- W3083424454
- Cited by
- 325
- References
- 45
- FWCI
- 12.58
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
