The DOI matches its Crossref record; 41 of the 43 references it cites were found in a registry or an index.
A Comparison of MCC and CEN Error Measures in Multi-Class Prediction
Open accessAbstract
We show that the Confusion Entropy, a measure of performance in multiclass problems has a strong (monotone) relation with the multiclass generalization of a classical metric, the Matthews Correlation Coefficient. Analytical results are provided for the limit cases of general no-information (n-face dice rolling) of the binary classification. Computational evidence supports the claim in the general case.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“MCC is the only binary classification rate that generates a high score only if the binary predictor was able to correctly predict the majority of positive data instances and the majority of negative data instances [80, 97].”
From The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation (2020) · cites this paper as [80]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
A Mathematical Theory of Communication
Shannon, C. E. (1948) · The MIT Press eBooks · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 50,387
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
The Proceedings of the National Academy of Sciences
Richards, J. W. (1915) · Science · doi:10.1126/science.42.1074.161 · Cited by 13,941
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 12,051
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515
The use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217
Repeated observation of breast tumor subtypes in independent gene expression data sets
Sørlie, T., Tibshirani, R. J., Parker, J. S., Hastie, T. J., Marron, J. S., Nobel, A. B., Deng, S., Johnsen, H., Pesich, R., Geisler, S. B., et al. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0932692100 · Cited by 5,508
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399 · cited 1×
“This latter function was introduced in (Matthews, 1975) and it is also known as the φ -coe ffi cient, corresponding for a 2 × 2 contingency table to the square root of the average χ 2 statistic χ 2 / n . p MCC has recently attracted the attention of the machine learning community (Baldi et al., 2000)…”
BackgroundA Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems
Hand, D. J., & Till, R. J. (2001) · Machine Learning · doi:10.1023/a:1010920819831 · Cited by 2,276
Using AUC and accuracy in evaluating learning algorithms
Huang, J., & Ling, C. X. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.50 · Cited by 2,148
Assessing the accuracy of prediction algorithms for classification: an overview
Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A. F., & Nielsen, H. (2000) · Bioinformatics · doi:10.1093/bioinformatics/16.5.412 · Cited by 2,111
IEEE Transactions on Knowledge and Data Engineering
(2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.69 · Cited by 1,671
García, S., & Herrera, F. (2008) · Cited by 1,277
Measuring classifier performance: a coherent alternative to the area under the ROC curve
Hand, D. (2009) · Machine-mediated learning · doi:10.1007/s10994-009-5119-5 · Cited by 1,116
Information Theory and Coding.
Good, I. J., & Abramson, N. (1964) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2343598 · Cited by 1,086
MAQC Consortium (2010) · Nature Biotechnology · doi:10.1038/nbt.1665 · Cited by 889
Self-citationAn experimental comparison of performance measures for classification
Ferri, C., Hernández-Orallo, J., & Modroiu, R. (2009) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.08.010 · Cited by 889
The Elements of Statistical Learning
Sapatinas, T. (2003) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2004.298_11.x · Cited by 635
Comparing two K-category assignments by a K-category correlation coefficient
Gorodkin, J. (2004) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2004.09.006 · Cited by 545
Proceedings of the International Joint Conference on Neural Networks (IJCNN) 2005
(2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1555790 · Cited by 382
Information Processing and Management
大貝, 明. (1984) · Journal of Information Processing and Management · doi:10.1241/johokanri.27.603 · Cited by 275
Volume under the ROC Surface for Multi-class Problems
Ferri, C., Hernández‐Orallo, J., & Salido, M. Á. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-39857-8_12 · Cited by 163
Evaluating diagnostic tests: The area under the ROC curve and the balance of errors
Hand, D. J. (2010) · Statistics in Medicine · doi:10.1002/sim.3859 · Cited by 133
Multi-class ROC analysis from a multi-objective optimisation perspective
Everson, R., & Fieldsend, J. E. (2005) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.016 · Cited by 122
Efficient Multiclass ROC Approximation by Decomposition via Confusion Matrix Perturbation Analysis
Landgrebe, T. C. W., & Duin, R. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70740 · Cited by 120
Proceedings of the 12th European Conference on Machine Learning
Raedt, L. D., & Flach, P. A. (2001) · Cited by 111
A novel measure for evaluating classifiers
Wei, J., Yuan, X., Hu, Q., & Wang, S. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.11.040 · Cited by 65
Information Theoretic Feature Crediting in Multiclass Support Vector Machines
Sindhwani, V., Bhattacharya, P., & Rakshit, S. (2001) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972719.16 · Cited by 63
Visualization and analysis of classifiers performance in multi-class medical data
Di̇ri̇, B., & Albayrak, S. (2006) · Expert Systems with Applications · doi:10.1016/j.eswa.2006.10.016 · Cited by 54
Shi, L. M., Campbell, G. R., Jones, W. D., Campagne, F., Wen, Z., Walker, S. J., Su, Z., Chu, T., Goodsaid, F. M., Pusztai, L., et al. (2010) · Cited by 37
Self-citationMulti-class AUC metrics and weighted alternatives
CALSTER, B. V., Belle, V. V., Condous, G., Bourne, T., Timmerman, D., & Huffel, S. V. (2008) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2008.4633979 · Cited by 24
Confusion Matrix Disagreement for Multiple Classifiers
Freitas, C. O. D. A., de Carvalho, J. M., Oliveira, J., Aires, S. B. K., & Sabourin, R. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-76725-1_41 · Cited by 20
A fast and efficient gene-network reconstruction method from multiple over-expression experiments
Stokić, D., Hanel, R., & Thurner, S. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-253 · Cited by 17
A method to quantify the error distribution in confusion matrices
van Son, R. J. J. H. (1995) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1995-518 · Cited by 13
Reconstructing Linear Gene Regulatory Networks
Supper, J., Spieth, C., & Zell, A. (2007) · Springer eBooks · doi:10.1007/978-3-540-71783-6_26 · Cited by 9
Comparing Classification Results between N-ary and Binary Problems
Felkin, M. (2007) · Studies in computational intelligence · doi:10.1007/978-3-540-44918-8_12 · Cited by 8
Li, Y. (2022) · Digital Archive @ GSU · doi:10.57709/1059733 · Cited by 5
Biochimica et biophysica acta. Protein structure
ScienceDirect (1981) · Cited by 5
Distance-based Disagreement Classifiers Combination
Freitas, C. O. D. A., de Carvalho, J. M., Oliveira, J. J., Aires, S. B. K., & Sabourin, R. (2007) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/ijcnn.2007.4371390 · Cited by 2
Research works printed but not linked: 2
- [11]Landgrebe T, Duin R (2005) On Neyman-Pearson optimisation for multiclass classifiers. In: Proceedings 16th Annual Symposium of the Pattern Recognition Association of South Africa. PRASA, pp. 165–170.
- [12]Landgrebe T, Duin R (2006) A simplified extension of the Area under the ROC to the multiclass domain. In: Proceedings 17th Annual Symposium of the Pattern Recognition Association of South Africa. PRASA, pp. 241–245.
Cited by
Sorted by citations
Chicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143
Self-citationSendra, A. L., Carrasco, A., Martín-Gómez, A. M., & Heras, A. D. L. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.02.023 · Cited by 1,259
Chicco, D., Tötsch, N., & Jurman, G. (2021) · BioData Mining · doi:10.1186/s13040-021-00244-z · Cited by 893
Self-citationSiamese Neural Networks: An Overview
Chicco, D. (2020) · Methods in molecular biology · doi:10.1007/978-1-0716-0826-5_3 · Cited by 726
Chicco, D., & Jurman, G. (2023) · BioData Mining · doi:10.1186/s13040-023-00322-4 · Cited by 737
Self-citation
Metadata
- Authors
- Jurman, G., Riccadonna, S., & Furlanello, C.
- Affiliations
- Fondazione Bruno Kessler (IT)
- Corresponding author
- Jurman, Giuseppe
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-08-08
- Volume
- 7
- Issue
- 8
- Pages
- e41882
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3414515
- OpenAlex
- W1988301066
- Cited by
- 406
- References
- 43
- Citations by year
- 2026: 17 · 2025: 29 · 2024: 38 · 2023: 54 · 2022: 47 · 2021: 54 · 2020: 32 · 2019: 36 · 2018: 25 · 2017: 23 · 2016: 15 · 2015: 12 · 2014: 14 · 2013: 10
- FWCI
- 11.34
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- European Commission
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
- Keywords
- Matthews correlation coefficient, multi-class classification, generalization, binary classification, monotonicity
- MeSH terms
- Algorithms, Artificial Intelligence, Computers, Genotype, Probability, ROC Curve, Software, Statistics as Topic, Reproducibility of Results, Models, Statistical, Entropy, Computational Biology, Area Under Curve, Oligonucleotide Array Sequence Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
