The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Information theoretic measures for clusterings comparison: is a correction for chance necessary?
Abstract
Information theoretic based measures form a fundamental class of similarity measures for comparing clusterings, beside the class of pair-counting based and set-matching based measures. In this paper, we discuss the necessity of correction for chance for information theoretic based measures for clusterings comparison. We observe that the baseline for such measures, i.e. average value between random partitions of a data set, does not take on a constant value, and tends to have larger variation when the ratio between the number of data points and the number of clusters is small. This effect is similar in some other non-information theoretic based measures such as the well-known Rand Index. Assuming a hypergeometric model of randomness, we derive the analytical formula for the expected mutual information value between a pair of clusterings, and then propose the adjusted version for several popular information theoretic based measures. Some examples are given to demonstrate the need and usefulness of the adjusted measures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
Hubert, L., & Arabie, P. (1985) · Journal of Classification · doi:10.1007/bf01908075 · Cited by 8,219
Objective Criteria for the Evaluation of Clustering Methods
Rand, W. M. (1971) · Journal of the American Statistical Association · doi:10.1080/01621459.1971.10482356 · Cited by 6,566
Monti, S., Tamayo, P., Mesirov, J., & Golub, T. (2003) · Machine Learning · doi:10.1023/a:1023949509487 · Cited by 2,116 · cited 2×
“In an era where a huge number of clustering algorithms exist, the Consensus Clustering idea (Monti et al., 2003; Strehl & Ghosh, 2002; Yu et al., 2007) has recently received increasing interest.”
BackgroundVinh, N. X., Epps, J., & Bailey, J. (2010) · Minerva Access (University of Melbourne) · doi:10.5555/1756006.1953024 · Cited by 1,781
Self-citation(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303321897735 · Cited by 1,092
Clustering on the Unit Hypersphere using von Mises-Fisher Distributions
Banerjee, A., Dhillon, I. S., Ghosh, J., & Sra, S. (2005) · MPG.PuRe (Max Planck Society) · doi:10.5555/1046920.1088718 · Cited by 766
Comparing clusterings: an axiomatic view
Meilǎ, M. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102424 · Cited by 589
Ord, K., & Lancaster, H. O. (1972) · International Statistical Review · doi:10.2307/1402483 · Cited by 389
Graph-based consensus clustering for class discovery from gene expression data
Yu, Z., Wong, H.-S., & Wang, H. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm463 · Cited by 177
Cited by
Algorithms for hierarchical clustering: an overview
Murtagh, F., & Contreras, P. (2012) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.53 · Cited by 1,874
Vinh, N. X., Epps, J., & Bailey, J. (2010) · Minerva Access (University of Melbourne) · doi:10.5555/1756006.1953024 · Cited by 1,781
Self-citationQuantifiable predictive features define epitope-specific T cell receptor repertoires
Dash, P., Fiore-Gartland, A. J., Hertz, T., Wang, G. C., Sharma, S., Souquette, A., Crawford, J. C., Clemens, E. B., Nguyen, T. H. O., Kedzierska, K., et al. (2017) · Nature · doi:10.1038/nature22383 · Cited by 897
Evaluating user privacy in bitcoin
Androulaki, E., Karame, G. O., Roeschlin, M., Scherer, T., & Capkun, S. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-39884-1_4 · Cited by 700
The ground truth about metadata and community detection in networks
Peel, L., Larremore, D. B., & Clauset, A. (2017) · Science Advances · doi:10.1126/sciadv.1602548 · Cited by 449
Multiple Kernel Fuzzy Clustering
Huang, H.-C., Chuang, Y.-Y., & Chen, C.-S. (2012) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2011.2170175 · Cited by 406
Human brain mapping: A systematic comparison of parcellation methods for the human cerebral cortex
Arslan, S., Ktena, S. I., Makropoulos, A., Robinson, E. C., Rueckert, D., & Parisot, S. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.014 · Cited by 374
“Intuitively, N00 and N11 account for the agreement of parcellations, whereas N01 and N10 indicate their disagreement (Vinh et al., 2009).”
BackgroundInfluentialXie, J., Gao, H., Xie, W., Liu, X., & Grant, P. W. (2016) · Information Sciences · doi:10.1016/j.ins.2016.03.011 · Cited by 360
The Effects of Data Quality on Machine Learning Performance on Tabular Data
Mohammed, S., Budach, L., Feuerpfeil, M., Ihde, N., Nathansen, A., Noack, N., Patzlaff, H., Naumann, F., & Harmouch, H. (2025) · Information Systems · doi:10.1016/j.is.2025.102549 · Cited by 331
“We use an adapted version of MI called the adjusted mutual information (AMI) [51].”
Which fMRI clustering gives good brain parcellations?
Thirion, B., Varoquaux, G., Dohmatob, E., & Poline, J.-B. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00167 · Cited by 327
GUNC: detection of chimerism and contamination in prokaryotic genomes
Orakov, A., Fullam, A., Coelho, L. P., Khedkar, S., Szklarczyk, D., Mende, D. R., Schmidt, T. S. B., & Bork, P. (2021) · Genome Biology · doi:10.1186/s13059-021-02393-0 · Cited by 299
Metrics for Community Analysis: A Survey
Chakraborty, T., Dalmia, A., Mukherjee, A., & Ganguly, N. (2018) · ACM Computing Surveys · doi:10.1145/3091106 · Cited by 296
Algorithms for hierarchical clustering: an overview,II
Murtagh, F., & Contreras, P. (2017) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1219 · Cited by 285
Misbehavior in Bitcoin: A Study of Double-Spending and Accountability
Karame, G. O., Androulaki, E., Roeschlin, M., Gervais, A., & Čapkun, S. (2015) · ACM Transactions on Information and System Security · doi:10.1145/2732196 · Cited by 186
Unifying the known and unknown microbial coding sequence space
Vanni, C., Schechter, M. S., Acinas, S. G., Barberán, A., Buttigieg, P. L., Casamayor, E. O., Delmont, T. O., Duarte, C. M., Eren, A. M., Finn, R. D., et al. (2022) · eLife · doi:10.7554/elife.67667 · Cited by 134
Characterizing networks of propaganda on twitter: a case study
Guarino, S., Trino, N., Celestini, A., Chessa, A., & Riotta, G. (2020) · Applied Network Science · doi:10.1007/s41109-020-00286-y · Cited by 49
Marin, E. C., Morris, B. J., Stürner, T., Champion, A. S., Krzeminski, D., Badalamente, G., Gkantia, M., Dunne, C. R., Eichler, K., Takemura, S.-Y., et al. (2024) · doi:10.7554/elife.97766.1 · Cited by 44
Marin, E. C., Morris, B. J., Stürner, T., Champion, A. S., Krzeminski, D., Badalamente, G., Gkantia, M., Dunne, C. R., Eichler, K., Takemura, S.-Y., et al. (2023) · doi:10.1101/2023.06.05.543407 · Cited by 33
The Impact of Random Models on Clustering Similarity
Gates, A. J., & Ahn, Y.-Y. (2017) · doi:10.1101/196840 · Cited by 24
The Geometry of Concepts: Sparse Autoencoder Feature Structure
Li, Y., Michaud, E. J., Baek, D. D., Engels, J., Sun, X., & Tegmark, M. (2024) · arXiv · doi:10.48550/arxiv.2410.19750 · Cited by 14
Marin, E. C., Morris, B. J., Stürner, T., Champion, A. S., Krzeminski, D., Badalamente, G., Gkantia, M., Dunne, C. R., Eichler, K., Takemura, S.-Y., et al. (2024) · eLife · doi:10.7554/elife.97766 · Cited by 13
Utgés, J. S., MacGowan, S. A., Ives, C. M., & Barton, G. J. (2024) · Communications Biology · doi:10.1038/s42003-024-05970-8 · Cited by 10
Metadata
- Authors
- Vinh, N. X., Epps, J., & Bailey, J.
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 1073-1080
- OpenAlex
- W2113054345
- Cited by
- 938
- References
- 13
- FWCI
- 18.57
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.