Adjusting for Chance Clustering Comparison Measures
Open accessThe DOI matches its OpenAlex record; 39 of the 39 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Possible generalization of Boltzmann-Gibbs statistics
Tsallis, C. (1988) · Journal of Statistical Physics · doi:10.1007/bf01016429 · Cited by 9,589 · cited 1×
“Generalized Information Theoretic (IT) measures based on the generalized Tsallis q -entropy (Tsallis, 1988) can be defined for random variables (Furuichi, 2006) and also be applied to the task of comparing partitions (Simovici, 2007).”
MethodObjective Criteria for the Evaluation of Clustering Methods
Rand, W. M. (1971) · Journal of the American Statistical Association · doi:10.1080/01621459.1971.10482356 · Cited by 6,566
On Measures of Entropy and Information
Rényi, A. (1961) · Project Euclid (Cornell University) · Cited by 4,031
Unbiased Recursive Partitioning: A Conditional Inference Framework
Hothorn, T., Hornik, K., & Zeileis, A. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x133933 · Cited by 4,021 · cited 2×
“This hypothesis has been employed in previous published research to study selection bias (White and Liu, 1994; Frank and Witten, 1998; Dobra and Gehrke, 2001; Shih, 2004; Hothorn et al., 2006; Romano et al., 2014).”
BackgroundMethodAnalyzing Neural Time Series Data: Theory and Practice
Cohen, M. X. (2014) · doi:10.7551/mitpress/9609.001.0001 · Cited by 2,105
Vinh, N. X., Epps, J., & Bailey, J. (2010) · Minerva Access (University of Melbourne) · doi:10.5555/1756006.1953024 · Cited by 1,781
Self-citationComparing clusterings—an information based distance
Meilă, M. (2007) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2006.11.013 · Cited by 1,546 · cited 1×
“This upper bound is valid for any q ∈ R + − { 1 } and allows us to link different existing measures as we will show in the next sections of the paper.”
BackgroundEstimation of entropy and mutual information
Paninski, L. (2003) · Neural Computation · doi:10.1162/089976603321780272 · Cited by 1,442
Introduction to Nonextensive Statistical Mechanics
(2009) · Springer eBooks · doi:10.1007/978-0-387-85359-8 · Cited by 1,426
A First Course in Probability.
Sherman, M. J., & Ross, S. (1984) · Journal of the American Statistical Association · doi:10.2307/2288753 · Cited by 1,344
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303321897735 · Cited by 1,092
Information theoretic measures for clusterings comparison: is a correction for chance necessary?
Vinh, N. X., Epps, J., & Bailey, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553511 · Cited by 938
Self-citation(1967) · Cited by 853
Data Clustering: Algorithms and Applications
(2018) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315373515 · Cited by 756
Criminisi, A. (2011) · now publishers, Inc. eBooks · doi:10.1561/9781601985415 · Cited by 703
Criminisi, A., Shotton, J., & Konukoglu, E. (2012) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000035 · Cited by 677 · cited 1×
“…the most similar partition to a reference ground truth partition: e.g., categorical feature selection (Vinh et al., 2014), decision tree induction (Criminisi et al., 2012), generation of alternative or multi-view clusterings (M¨uller et al., 2013), or the exploration of the clustering space with…”
BackgroundA stability based method for discovering structure in clustered data
Ben-Hur, A., Elisseeff, A., & Guyon, I. (2001) · Biocomputing 2002 · doi:10.1142/9789812799623_0002 · Cited by 555 · cited 2×
“This category comprises the well known similarity measures Rand Index (RI) (Rand, 1971) and the Jaccard coefficient (J) (Ben-Hur et al., 2001).”
BackgroundGeneralized information functions
Daróczy, Z. (1970) · Information and Control · doi:10.1016/s0019-9958(70)80040-7 · Cited by 495 · cited 2×
“proposed in Havrda and Charvát (1967); Daróczy (1970). Results available in literature about these generalized entropies are equivalently valid for all the proposed versions.”
BackgroundMethodAdapting the right measures for K-means clustering
Wu, J., Xiong, H., & Chen, J. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557115 · Cited by 255 · cited 2×
“Given the variety of different possible measures, it is very challenging to identify the best choice for a particular application scenario (Wu et al., 2009).”
BackgroundCaruana, R., Elhawary, M., Nguyen, N., & Smith, C. (2006) · Sixth International Conference on Data Mining (ICDM'06) · doi:10.1109/icdm.2006.103 · Cited by 180 · cited 2×
“ion tree induction (Criminisi et al., 2012), generation of alternative or multi-view clusterings (Muller et al., 2013), or the exploration of the clustering space with the Meta-Clustering algorithm (Caruana et al., 2006; Lei et al., 2014) to list a few. Dierent values for qin AMI q yield to dierent biases. The source…”
BackgroundMethodOn the Equivalence of Cohen’s Kappa and the Hubert-Arabie Adjusted Rand Index
Warrens, M. J. (2008) · Journal of Classification · doi:10.1007/s00357-008-9023-7 · Cited by 160
On Similarity Indices and Correction for Chance Agreement
Albatineh, A. N., Niewiadomska-Bugaj, M., & Mihalko, D. (2006) · Journal of Classification · doi:10.1007/s00357-006-0017-z · Cited by 160 · cited 5×
“The authors of (Albatineh et al., 2006) noticed that the difference between the approximation and the expected value obtained with the hypergeometric model is small on empirical experiments when N is large.”
BackgroundMethodResultInfluentialMorey, L. C., & Agresti, A. (1984) · Educational and Psychological Measurement · doi:10.1177/0013164484441003 · Cited by 159
Effective global approaches for mutual information based feature selection
Nguyen, X. V., Chan, J., Romano, S., & Bailey, J. (2014) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623611 · Cited by 152 · cited 1×
“, categorical feature selection (Vinh et al., 2014), decision tree induction (Criminisi et al.”
BackgroundSelf-citationInformation theoretical properties of Tsallis entropiesPreprint
Furuichi, S. (2006) · Cited by 123
Is Normalized Mutual Information a Fair Measure for Comparing Community Detection Methods?
Amelio, A., & Pizzuti, C. (2015) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/2808797.2809344 · Cited by 102 · cited 1×
“For example, it has been argued that in some cases the selection of clustering solutions should be biased towards solutions with the same number of clusters as in the reference (Amelio and Pizzuti, 2015).”
BackgroundNonextensive Information Theoretic Kernels on Measures
Martins, A. F. T., Smith, N. A., Xing, E. P., Aguiar, P. M. Q., & Figueiredo, M. A. T. (2009) · Cited by 93
Romano, S., Bailey, J., Vinh, N. X., & Verspoor, K. M. (2014) · Cited by 83
Self-citationDiscovering Multiple Clustering Solutions: Grouping Objects in Different Views of the Data
Müller, E., Günnemann, S., Färber, I., & Seidl, T. (2012) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2012.142 · Cited by 71 · cited 1×
“…feature selection (Vinh et al., 2014), decision tree induction (Criminisi et al., 2012), generation of alternative or multi-view clusterings (M¨uller et al., 2013), or the exploration of the clustering space with the Meta-Clustering algorithm (Caruana et al., 2006; Lei et al., 2014) to…”
MethodCorrecting Jaccard and other similarity indices for chance agreement in cluster analysis
Albatineh, A. N., & Niewiadomska–Bugaj, M. (2011) · Advances in Data Analysis and Classification · doi:10.1007/s11634-011-0090-y · Cited by 70
Using a Permutation Test for Attribute Selection in Decision Trees
Frank, E., & Witten, I. H. (1998) · Research Commons (University of Waikato) · Cited by 55
A note on split selection bias in classification trees
Shih, Y. (2003) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(03)00064-1 · Cited by 55 · cited 3×
“Researchers in this area agree that in order to achieve unbiased selection of partitions, distribution properties of similarity measures have to be taken into account (Dobra and Gehrke, 2001; Shih, 2004; Hothorn et al., 2006).”
BackgroundMethodBayesian and Quasi-Bayesian Estimators for Mutual Information from Discrete Data
Archer, E., Park, I. M., & Pillow, J. W. (2013) · Entropy · doi:10.3390/e15051738 · Cited by 50 · cited 1×
“It is indeed well known that information theoretic measures are severely biased when plug-in estimators are used, and many have worked on correcting this bias for decades: there exist in literature frequentist approaches (Paninski, 2003) as well as Bayesian approaches (Archer et al., 2013; Cerquetti, 2014) to reduce b…”
BackgroundInference of gene regulatory networks from time series by Tsallis entropy
Lopes, F. M., de Oliveira, E. A., & Junior, R. M. C. (2011) · BMC Systems Biology · doi:10.1186/1752-0509-5-61 · Cited by 45
Bias Correction in Classification Tree Construction
Dobra, A., & Gehrke, J. E. (2001) · International Conference on Machine Learning · Cited by 36
A Framework to Adjust Dependency Measure Estimates for Chance
Romano, S., Vinh, N. X., Bailey, J., & Verspoor, K. M. (2016) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611974348.48 · Cited by 18 · cited 1×
“Although the choice of whether performing standardization or not is dependent to the application (Romano et al., 2015).”
BackgroundSelf-citationOn Generalized Entropy and Entropic Metrics.
Simovici, D. A. (2007) · Cited by 9
Bayesian nonparametric estimation of Tsallis diversity indices under Gnedin-Pitman priorsPreprint
Cerquetti, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.3441 · Cited by 5 · cited 1×
“It is indeed well known that information theoretic measures are severely biased when plug-in estimators are used, and many have worked on correcting this bias for decades: there exist in literature frequentist approaches (Paninski, 2003) as well as Bayesian approaches (Archer et al., 2013; Cerquetti, 2014) to reduce b…”
BackgroundImproving Decision Trees Using Tsallis Entropy.Preprint
Wang, Y., Song, C., & Xia, S. (2015) · arXiv (Cornell University) · Cited by 1
Cited by
Chounta, I.-A., Bardone, E., Raudsep, A., & Pedaste, M. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00243-5 · Cited by 441
Curiskis, S. A., Drake, B., Osborn, T. R., & Kennedy, P. J. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2019.04.002 · Cited by 304
Zheng, R., Li, M., Liang, Z., Wu, F.-X., Pan, Y., & Wang, J. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz139 · Cited by 169
The Impact of Random Models on Clustering Similarity
Gates, A. J., & Ahn, Y.-Y. (2017) · doi:10.1101/196840 · Cited by 24
Metadata
- Authors
- Romano, S., Vinh, N. X., Bailey, J., & Verspoor, K. M.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2015-12-03
- Volume
- 17
- Issue
- 1
- Pages
- 4635-4666
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1512.01286
- OpenAlex
- W2963083182
- Cited by
- 192
- References
- 39
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.