Cited by 406Imbalanced Data Classification TechniquesMachine Learning and Data ClassificationData Mining and Machine Learning Applications

The DOI matches its Crossref record; 41 of the 43 references it cites were found in a registry or an index.

A Comparison of MCC and CEN Error Measures in Multi-Class Prediction

Open access

Jurman, Giuseppe, Riccadonna, Samantha, Furlanello, Cesare·2012·PLoS ONE

DOI 10.1371/journal.pone.0041882PMC PMC3414515

Open this paper in CoChat — free account

Abstract

We show that the Confusion Entropy, a measure of performance in multiclass problems has a strong (monotone) relation with the multiclass generalization of a classical metric, the Matthews Correlation Coefficient. Analytical results are provided for the limit cases of general no-information (n-face dice rolling) of the binary classification. Computational evidence supports the claim in the general case.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0041882

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Figure 1 Examples of CEN and MCC for different confusion matrices.CC BY · PubMed Central

Figure 2 Dotplots of CEN versus MCC. (a) and CEN versus tMCC (b) for 200 000 random confusion matrices of different dimensions.CC BY · PubMed Central

Figure 3 Lines describing CEN and MCC of a confusion matrix for increasing ratio . Gray vertical lines correspond to the examples provided in Fig. 1, Box 4.CC BY · PubMed Central

Figure 4 Scatter plots of CEN versus MCC for all the confusion matrices of binary classification tasks with s = 5,10,50,75 samples and for the cumulative set of all 4 598 125 matrices with .CC BY · PubMed Central

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
A Mathematical Theory of Communication
Shannon, C. E. (1948) · The MIT Press eBooks · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 50,387
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
The Proceedings of the National Academy of Sciences
Richards, J. W. (1915) · Science · doi:10.1126/science.42.1074.161 · Cited by 13,941
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 12,051
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515
The use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217
Repeated observation of breast tumor subtypes in independent gene expression data sets
Sørlie, T., Tibshirani, R. J., Parker, J. S., Hastie, T. J., Marron, J. S., Nobel, A. B., Deng, S., Johnsen, H., Pesich, R., Geisler, S. B., et al. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0932692100 · Cited by 5,508
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399 · cited 1×
“This latter function was introduced in (Matthews, 1975) and it is also known as the φ -coe ﬃ cient, corresponding for a 2 × 2 contingency table to the square root of the average χ 2 statistic χ 2 / n . p MCC has recently attracted the attention of the machine learning community (Baldi et al., 2000)…”
Background

Show all 43 references

Research works printed but not linked: 2

[11]Landgrebe T, Duin R (2005) On Neyman-Pearson optimisation for multiclass classifiers. In: Proceedings 16th Annual Symposium of the Pattern Recognition Association of South Africa. PRASA, pp. 165–170.
[12]Landgrebe T, Duin R (2006) A simplified extension of the Area under the ROC to the multiclass domain. In: Proceedings 17th Annual Symposium of the Pattern Recognition Association of South Africa. PRASA, pp. 241–245.

Sorted by citations

The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation
Chicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143
Self-citation
The impact of class imbalance in classification performance metrics based on the binary confusion matrix
Sendra, A. L., Carrasco, A., Martín-Gómez, A. M., & Heras, A. D. L. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.02.023 · Cited by 1,259
The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation
Chicco, D., Tötsch, N., & Jurman, G. (2021) · BioData Mining · doi:10.1186/s13040-021-00244-z · Cited by 893
Self-citation
Siamese Neural Networks: An Overview
Chicco, D. (2020) · Methods in molecular biology · doi:10.1007/978-1-0716-0826-5_3 · Cited by 726
The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification
Chicco, D., & Jurman, G. (2023) · BioData Mining · doi:10.1186/s13040-023-00322-4 · Cited by 737
Self-citation
Machine learning can predict survival of patients with heart failure from serum creatinine and ejection fraction alone
Chicco, D., & Jurman, G. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-1023-5 · Cited by 664
Self-citation
Error Metrics and Performance Fitness Indicators for Artificial Intelligence and Machine Learning in Engineering and Sciences
Naser, M. Z., & Alavi, A. H. (2021) · Architecture Structures and Construction · doi:10.1007/s44150-021-00015-8 · Cited by 554
Boosting methods for multi-class imbalanced data classification: an experimental review
Tanha, J., Abdi, Y., Samadi, N., Razzaghi-Asl, N., & Asadpour, M. (2020) · Journal Of Big Data · doi:10.1186/s40537-020-00349-y · Cited by 497
The Matthews Correlation Coefficient (MCC) is More Informative Than Cohen’s Kappa and Brier Score in Binary Classification Assessment
Chicco, D., Warrens, M. J., & Jurman, G. (2021) · IEEE Access · doi:10.1109/access.2021.3084050 · Cited by 503
Self-citation
A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification
Al–antari, M. A., Al‐masni, M. A., Choi, M., Han, S., & Kim, T. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.06.003 · Cited by 417
Breast cancer histopathological image classification using convolutional neural networks with small SE-ResNet module
Jiang, Y., Chen, L., Zhang, H., & Xiao, X. (2019) · PLoS ONE · doi:10.1371/journal.pone.0214587 · Cited by 386
Multivariate comparison of classification performance measures
Ballabio, D., Grisoni, F., & Todeschini, R. (2017) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2017.12.004 · Cited by 331
Why Cohen’s Kappa should be avoided as performance measure in classification
Delgado, R., & Tibau, X.-A. (2019) · PLOS ONE · doi:10.1371/journal.pone.0222916 · Cited by 324
DNA methylation-based predictors of health: applications and statistical considerations
Yousefi, P., Suderman, M. J., Langdon, R., Whitehurst, O., Smith, G. D., & Relton, C. L. (2022) · Nature Reviews Genetics · doi:10.1038/s41576-022-00465-w · Cited by 233
100% Classification Accuracy Considered Harmful: The Normalized Information Transfer Factor Explains the Accuracy Paradox
Valverde-Albacete, F. J., & Peláez-Moreno, C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0084217 · Cited by 212
A mountable toilet system for personalized health monitoring via the analysis of excreta
Park, S.-M., Won, D. D., Lee, B. J., Escobedo, D., Esteva, A., Aalipour, A., Ge, T. J., Kim, J. H., Suh, S., Choi, E. H., et al. (2020) · Nature Biomedical Engineering · doi:10.1038/s41551-020-0534-9 · Cited by 175
Textural differences between renal cell carcinoma subtypes: Machine learning-based quantitative computed tomography texture analysis with independent external validation
Koçak, B., Yardımcı, A. H., Bektaş, C. T., Türkcanoğlu, M. H., Erdim, Ç., Yücetaş, U., Koca, S. B., & Kılıçkesmez, Ö. (2018) · European Journal of Radiology · doi:10.1016/j.ejrad.2018.08.014 · Cited by 135
Classification of Heart Sounds Using Convolutional Neural Network
Li, F., Tang, H. X., Shang, S., Mathiak, K., & Cong, F. (2020) · Applied Sciences · doi:10.3390/app10113956 · Cited by 140
Microstructural characteristics and their impact on mechanical properties of steel-PVA fiber reinforced concrete
Liu, F., Xu, K., Ding, W., Qiao, Y., & Wang, L. (2021) · Cement and Concrete Composites · doi:10.1016/j.cemconcomp.2021.104196 · Cited by 117
The Benefits of the Matthews Correlation Coefficient (MCC) Over the Diagnostic Odds Ratio (DOR) in Binary Classification Assessment
Chicco, D., Старовойтов, В. В., & Jurman, G. (2021) · IEEE Access · doi:10.1109/access.2021.3068614 · Cited by 112
Self-citation
Deconstructing cross-entropy for probabilistic binary classifiers
Ramos, D., Franco-Pedroso, J., Lozano-Diez, A., & Gonzalez-Rodriguez, J. (2018) · Entropy · doi:10.3390/e20030208 · Cited by 99
Real-world application of machine-learning-based fault detection trained with experimental data
Bode, G., Thul, S., Baranski, M., & Müller, D. (2020) · Energy · doi:10.1016/j.energy.2020.117323 · Cited by 99
Prediction of N-linked glycosylation sites using position relative features and statistical moments
Akmal, M. A., Rasool, N., & Khan, Y. D. (2017) · PLoS ONE · doi:10.1371/journal.pone.0181966 · Cited by 99
Evaluation of single-cell classifiers for single-cell RNA sequencing data sets
Zhao, X. S., Wu, S., Fang, N., Sun, X. S., & Fan, J. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz096 · Cited by 102
Semantic Frames to Predict Stock Price Movement
Xie, B., Passonneau, R. J., Wu, L., & Creamer, G. G. (2013) · Cited by 100
A study on volatile organic compounds emitted by in-vitro lung cancer cultured cells using gas sensor array and SPME-GCMS
Thriumani, R., Zakaria, A., Hashim, Y. Z. H.-Y., Jeffree, A. I., Helmy, K., Kamarudin, L. M., Omar, M. I., Shakaff, A. Y. M., Adom, A. H., & Persaud, K. C. (2018) · BMC Cancer · doi:10.1186/s12885-018-4235-7 · Cited by 89
Gene expression profiling gut microbiota in different races of humans
Chen, L., Zhang, Y., Huang, T., & Cai, Y. (2016) · Scientific Reports · doi:10.1038/srep23075 · Cited by 89
Detecting Drinking Episodes in Young Adults Using Smartphone-based Sensors
Bae, S. W., Ferreira, D., Suffoletto, B. P., Puyana, J. C., Kurtz, R., Chung, T. A., & Dey, A. K. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3090051 · Cited by 84
Classifiers and their Metrics Quantified
Brown, J. B. (2018) · Molecular Informatics · doi:10.1002/minf.201700127 · Cited by 82
Making Disk Failure Predictions SMARTer
Lu, S., Luo, B., Patel, T., Yao, Y., Tiwari, D., & Shi, W. (2020) · File and Storage Technologies · Cited by 81
In vivo Raman spectroscopy for detection of oral neoplasia: A pilot clinical study
Krishna, H., Majumder, S. K., Chaturvedi, P., Muttagi, S., & Gupta, P. K. (2013) · Journal of Biophotonics · doi:10.1002/jbio.201300030 · Cited by 81
Real-time abnormality detection and classification in diesel engine operations with convolutional neural network
Shahid, S. M., Ko, S.-H., & Kwon, S. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116233 · Cited by 80
Heterogeneous Ensemble Combination Search Using Genetic Algorithm for Class Imbalanced Data Classification
Haque, M. N., Noman, N., Berretta, R. E., & Moscato, P. (2016) · PLoS ONE · doi:10.1371/journal.pone.0146116 · Cited by 77
Machine Learning-Based Ensemble Recursive Feature Selection of Circulating miRNAs for Cancer Tumor Classification
Lopez‐Rincon, A., Mendoza‐Maldonado, L., Martínez‐Archundia, M., Schönhuth, A., Kraneveld, A. D., Garssen, J., & Tonda, A. P. (2020) · Cancers · doi:10.3390/cancers12071785 · Cited by 76
Identifying maximum imbalance in datasets for fault diagnosis of gearboxes
dos Santos, P. T. C., Maudes, J., & Bustillo, A. (2015) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-015-1110-0 · Cited by 75
Integrative Network Fusion: A Multi-Omics Approach in Molecular Profiling
Chierici, M., Bussola, N., Marcolini, A., Francescatto, M., Zandonà, A., Trastulla, L., Agostinelli, C., Jurman, G., & Furlanello, C. (2020) · Frontiers in Oncology · doi:10.3389/fonc.2020.01065 · Cited by 82
Self-citation
Prosodic analysis of neutral, stress-modified and rhymed speech in patients with Parkinson's disease
Galáž, Z., Mekyska, J., Mzourek, Z., Smékal, Z., Rektorová, I., Eliasova, I., Košťálová, M., Mračková, M., & Beránková, D. (2016) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2015.12.011 · Cited by 72
HiTSelect: a comprehensive tool for high-complexity-pooled screen analysis
Diaz, A. A., Qin, H., Ramalho‐Santos, M., & Song, J. S. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku1197 · Cited by 66
First Multitarget Chemo-Bioinformatic Model To Enable the Discovery of Antibacterial Peptides against Multiple Gram-Positive Pathogens
Speck‐Planche, A., Kleandrova, V. V., Ruso, J. M., & Cordeiro, M. N. D. S. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00630 · Cited by 64
Predicting Delivery Capability in Iterative Software Development
Choetkiertikul, M., Dam, H. K., Tran, T., Ghose, A., & Grundy, J. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2693989 · Cited by 64
StutterNet: Stuttering Detection Using Time Delay Neural Network
Sheikh, S. A., Sahidullah, M., Hirsch, F., & Ouni, S. (2021) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco54536.2021.9616063 · Cited by 60
Classifying Ten Types of Major Cancers Based on Reverse Phase Protein Array Profiles
Zhang, P., Chen, L., Huang, T., Zhang, N., Kong, X., & Cai, Y. (2015) · PLoS ONE · doi:10.1371/journal.pone.0123147 · Cited by 60
Evaluating reproducibility of AI algorithms in digital pathology with DAPPER
Bizzego, A., Bussola, N., Chierici, M., Maggio, V., Francescatto, M., Cima, L., Cristoforetti, M., Jurman, G., & Furlanello, C. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006269 · Cited by 60
Self-citation
Predicting the Slump of Industrially Produced Concrete from Mixture Proportions: Insights from Statistical Analysis, Data Visualization and Machine Learning
Zhang, X., Akber, M. Z., & Zheng, W. (2022) · Journal of Building Engineering · doi:10.1016/j.jobe.2022.104997 · Cited by 59
A statistical comparison between Matthews correlation coefficient (MCC), prevalence threshold, and Fowlkes–Mallows index
Chicco, D., & Jurman, G. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104426 · Cited by 55
Self-citation
Differentiating Between Multiple Myeloma and Metastasis Subtypes of Lumbar Vertebra Lesions Using Machine Learning–Based Radiomics
Xiong, X., Wang, J., Hu, S., Dai, Y., Zhang, Y., & Hu, C. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2021.601699 · Cited by 52
Integrating deep and radiomics features in cancer bioimaging
Bizzego, A., Bussola, N., Salvalai, D., Chierici, M., Maggio, V., Jurman, G., & Furlanello, C. (2019) · IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) · doi:10.1109/cibcb.2019.8791473 · Cited by 56
Self-citation
Generating and evaluating synthetic data in digital pathology through diffusion models
Pozzi, M., Noei, S., Robbi, E., Cima, L., Moroni, M., Munari, E., Torresani, E., & Jurman, G. (2024) · Scientific Reports · doi:10.1038/s41598-024-79602-w · Cited by 56
Self-citation
A medical diagnostic tool based on radial basis function classifiers and evolutionary simulated annealing
Alexandridis, A., & Chondrodima, E. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.03.008 · Cited by 47
Water quality prediction: Multi objective genetic algorithm coupled artificial neural network based approach
Chatterjee, S., Sarkar, S., Dey, N., Sen, S., Goto, T., & Debnath, N. C. (2017) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin.2017.8104902 · Cited by 47
Enhancing energy efficiency in the residential sector with smart meter data analytics
Hopf, K., Sodenkamp, M., & Staake, T. (2018) · Electronic Markets · doi:10.1007/s12525-018-0290-9 · Cited by 47
Pear Defect Detection Method Based on ResNet and DCGAN
Zhang, Y., Wa, S., Sun, P., & Wang, Y. (2021) · Information · doi:10.3390/info12100397 · Cited by 46
In-situ microtomography image segmentation for characterizing strain-hardening cementitious composites under tension using machine learning
Xu, K., Jin, Q., Li, J., Ushizima, D. M., Li, V. C., Kurtis, K. E., & Monteiro, P. J. M. (2023) · Cement and Concrete Research · doi:10.1016/j.cemconres.2023.107164 · Cited by 48
The MCC-F1 curve: a performance evaluation technique for binary classificationPreprint
Cao, C., Chicco, D., & Hoffman, M. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11278 · Cited by 45
An Invitation to Greater Use of Matthews Correlation Coefficient in Robotics and Artificial Intelligence
Chicco, D., & Jurman, G. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.876814 · Cited by 44
Self-citation
Comparison of High-Level Microarray Analysis Methods in the Context of Result Consistency
Chromiński, K., & Tkacz, M. A. (2015) · PLoS ONE · doi:10.1371/journal.pone.0128845 · Cited by 44
Deep Learning for Cross-Diagnostic Prediction of Mental Disorder Diagnosis and Prognosis Using Danish Nationwide Register and Genetic Data
Allesøe, R. L., Thompson, W. K., Bybjerg‐Grauholm, J., HOUGAARD, D. M., Nordentoft, M., Werge, T. M., Rasmussen, S., & Benros, M. E. (2022) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2022.4076 · Cited by 47
Pearson–Matthews correlation coefficients for binary and multinary classification
Stoica, P., & Babu, P. (2024) · Signal Processing · doi:10.1016/j.sigpro.2024.109511 · Cited by 43
Machine Learning and Artificial Intelligence for Infectious Disease Surveillance, Diagnosis, and Prognosis
Cheah, B. C. J., Vicente, C. R., & Chan, K. R. (2025) · Viruses · doi:10.3390/v17070882 · Cited by 61
Detection of wheezes using their signature in the spectrogram space and musical features
Mendes, L. G. A., Vogiatzis, I., Perantoni, E., Kaimakamis, E., Chouvarda, I., Maglaveras, N., Tsara, V., Teixeira, C., Carvalho, P., Henriques, J. M. O., et al. (2015) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7319657 · Cited by 43
Chemoinformatics for Medicinal Chemistry: In Silico Model to Enable the Discovery of Potent and Safer Anti-Cocci Agents
Speck‐Planche, A., & Cordeiro, M. N. D. S. (2014) · Future Medicinal Chemistry · doi:10.4155/fmc.14.136 · Cited by 44
A Novel Hybrid Classification Model of Genetic Algorithms, Modified k-Nearest Neighbor and Developed Backpropagation Neural Network
Salari, N., Shohaimi, S., Najafi, F., Nallappan, M., & Karishnarajah, I. (2014) · PLoS ONE · doi:10.1371/journal.pone.0112987 · Cited by 43
Learning From Highly Confident Samples for Automatic Knee Osteoarthritis Severity Assessment: Data From the Osteoarthritis Initiative
Wang, Y., Bi, Z., Xie, Y., Wu, T., Zeng, X., Chen, S., & Zhou, D. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3102090 · Cited by 44
Learning from adversarial medical images for X-ray breast mass segmentation
Shen, T., Gou, C., Wang, F., He, Z., & Chen, W. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.105012 · Cited by 41
Disparity in Frontal Lobe Connectivity on a Complex Bimanual Motor Task Aids in Classification of Operator Skill Level
Andreu-Pérez, J., Leff, D. R., Shetty, K., Darzi, A. W., & Yang, G. (2016) · Brain Connectivity · doi:10.1089/brain.2015.0350 · Cited by 40
Large expert-curated database for benchmarking document similarity detection in biomedical literature search
Brown, P., Tan, A.-C., El-Esawi, M. A., Liehr, T., Blanck, O., Gladue, D. P., Almeida, G. M. D. F., Cernava, T., Sorzano, C. Ó. S., Yeung, A. W. K., et al. (2019) · Database · doi:10.1093/database/baz085 · Cited by 48
Study of Multi-Class Classification Algorithms’ Performance on Highly Imbalanced Network Intrusion Datasets
Bulavas, V., Marcinkevičius, V., & Rumiński, J. (2021) · Informatica · doi:10.15388/21-infor457 · Cited by 44
Underwater object detection and temporal signal detection in turbid water using 3D-integral imaging and deep learning
Joshi, R. C., Usmani, K., Krishnan, G. S., Blackmon, F. A., & Javidi, B. (2023) · Optics Express · doi:10.1364/oe.510681 · Cited by 40
A comparative analysis of machine learning models for quality pillar assessment of SaaS services by multi-class text classification of users’ reviews
Raza, M. A., Hussain, F. K., Hussain, F. K., Zhao, M. H., & Rehman, Z. U. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.06.022 · Cited by 41
A Multi-Level Classification Approach for Sleep Stage Prediction With Processed Data Derived From Consumer Wearable Activity Trackers
Liang, Z., & Martell, M. A. C. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.665946 · Cited by 40
Privacy-Preserving Household Characteristic Identification With Federated Learning Method
Lin, J., Ma, J., & Zhu, J. G. (2021) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2021.3125677 · Cited by 41
Daily Locomotion Recognition and Prediction: A Kinematic Data-Based Machine Learning Approach
Figueiredo, J., Carvalho, S. P., Goncalve, D., Moreno, J. C., & Santos, C. P. (2020) · IEEE Access · doi:10.1109/access.2020.2971552 · Cited by 37
Predicting building age from urban form at large scale
Nachtigall, F., Milojevic-Dupont, N., Wagner, F., & Creutzig, F. (2023) · Computers Environment and Urban Systems · doi:10.1016/j.compenvurbsys.2023.102010 · Cited by 37
Establishing a common database of ice experiments and using machine learning to understand and predict ice behavior
Kellner, L., Stender, M., Polach, F. V. B. U., Herrnring, H., Ehlers, S., Hoffmann, N., & Høyland, K. V. (2019) · Cold Regions Science and Technology · doi:10.1016/j.coldregions.2019.02.007 · Cited by 37
Young driver fatal motorcycle accident analysis by jointly maximizing accuracy and information
Halbersberg, D., & Lerner, B. (2019) · Accident Analysis & Prevention · doi:10.1016/j.aap.2019.04.016 · Cited by 37
A Pan-Cancer Approach to Predict Responsiveness to Immune Checkpoint Inhibitors by Machine Learning
Polano, M., Chierici, M., Bo, M. D., Gentilini, D., Cintio, F. D., Baboçi, L., Gibbs, D. L., Furlanello, C., & Toffoli, G. (2019) · Cancers · doi:10.3390/cancers11101562 · Cited by 37
Self-citation
Valuing free-form text data from maintenance logs through transfer learning with CamemBERT
Cadavid, J. P. U., Grabot, B., Lamouri, S., Pellerin, R., & Fortin, A. (2020) · Enterprise Information Systems · doi:10.1080/17517575.2020.1790043 · Cited by 38
Deep Neural Networks and Transfer Learning on a Multivariate Physiological Signal Dataset
Bizzego, A., Gabrieli, G., & Esposito, G. (2021) · Bioengineering · doi:10.3390/bioengineering8030035 · Cited by 38
Noninvasive Diagnostic for COVID-19 from Saliva Biofluid via FTIR Spectroscopy and Multivariate Analysis
Nascimento, M. H. C., Marcarini, W. D., Folli, G. S., Filho, W. G. D. S., Barbosa, L. L., de Paulo, E. H., Vassallo, P. F., Mill, J. G., Baraúna, V. G., Martin, F. L., et al. (2022) · Analytical Chemistry · doi:10.1021/acs.analchem.1c04162 · Cited by 37
A computer-aided diagnosis system for breast ultrasound based on weighted BI-RADS classes
Rodríguez-Cristerna, A., Gómez‐Flores, W., & Pereira, W. C. D. A. (2017) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2017.10.004 · Cited by 36
Host Taxon Predictor - A Tool for Predicting Taxon of the Host of a Newly Discovered Virus
Gałan, W., Bąk, M., & Jakubowska, M. (2019) · Scientific Reports · doi:10.1038/s41598-019-39847-2 · Cited by 36
On Mathews Correlation Coefficient and Improved Distance Map Loss for Automatic Glacier Calving Front Segmentation in SAR Imagery
Davari, A., Islam, S., Seehaus, T., Hartmann, A., Braun, M., Maier, A., & Christlein, V. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3115883 · Cited by 35
Radiomics Model Based on Non-Contrast CT Shows No Predictive Power for Complete Pathological Response in Locally Advanced Rectal Cancer
Hamerla, G., Meyer, H., Hambsch, P. K., Wolf, U., Kuhnt, T., Hoffmann, K., & Surov, A. (2019) · Cancers · doi:10.3390/cancers11111680 · Cited by 34
Prediction of kinase inhibitors binding modes with machine learning and reduced descriptor sets
Abdelbaky, I., Tayara, H., & Chong, K. T. (2021) · Scientific Reports · doi:10.1038/s41598-020-80758-4 · Cited by 35
A molecular barcode and web-based data analysis tool to identify imported Plasmodium vivax malaria
Trimarsanto, H., Amato, R., Pearson, R. D., Sutanto, E., Noviyanti, R., Trianty, L., Marfurt, J., Pava, Z., Echeverry, D. F., Lopera-Mesa, T. M., et al. (2022) · Communications Biology · doi:10.1038/s42003-022-04352-2 · Cited by 34
Comparative study of quality estimation of binary classification
Старовойтов, В. В., & Golub, Y. I. (2020) · Informatics · doi:10.37661/1816-0301-2020-17-1-87-101 · Cited by 33
Prediction of fatigue failure in small-scale butt-welded joints with explainable machine learning
Braun, M., Kellner, L., Schreiber, S., & Ehlers, S. (2022) · Procedia Structural Integrity · doi:10.1016/j.prostr.2022.03.019 · Cited by 33
A hierarchical fusion framework to integrate homogeneous and heterogeneous classifiers for medical decision-making
Wang, L., Mo, T., Wang, X., Chen, W., He, Q., Li, X., Zhang, S., Yang, R., Wu, J., Gu, X., et al. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106517 · Cited by 32
Cardiac Diagnostic Feature and Demographic Identification (CDF-DI): An IoT Enabled Healthcare Framework Using Machine Learning
Kumar, D., Verma, C., Dahiya, S., Singh, P. K., Raboacă, M. S., Illés, Z., & Bakariya, B. (2021) · Sensors · doi:10.3390/s21196584 · Cited by 32
Instance space analysis and algorithm selection for the job shop scheduling problem
Strassl, S., & Musliu, N. (2022) · Computers & Operations Research · doi:10.1016/j.cor.2021.105661 · Cited by 32
Automated explainable deep learning framework for multiclass skin cancer detection and classification using hybrid YOLOv8 and vision transformer (ViT)
AbuAlkebash, H., Saleh, R. A. A., & Ertunç, H. M. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107934 · Cited by 31
Advanced Parametrization of Graphomotor Difficulties in School-Aged Children
Galáž, Z., Mucha, J., Zvončák, V., Mekyska, J., Smékal, Z., Šafárová, K., Ondrackova, A., Urbánek, T., Havigerová, J. M., Bednářová, J., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3003214 · Cited by 33
The flood cooking book: ingredients and regional flavors of floods across Germany
Merz, R., Tarasova, L., & Basso, S. (2020) · Environmental Research Letters · doi:10.1088/1748-9326/abb9dd · Cited by 30
Exploration of Convolutional Neural Network models for source code classification
Barchi, F., Parisi, E., Urgese, G., Ficarra, E., & Acquaviva, A. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.104075 · Cited by 35
Fuzzy Logic Determination of Lithologies from Well Log Data: Application to the KTB Project Data set (Germany)
Bosch, D. D., Ledo, J., & Queralt, P. (2013) · Surveys in Geophysics · doi:10.1007/s10712-013-9242-2 · Cited by 30
Identification of soil profile classes using depth-weighted visible–near-infrared spectral reflectance
Xie, X., & Li, A. (2018) · Geoderma · doi:10.1016/j.geoderma.2018.03.029 · Cited by 30
Robust Deep-Learning-Based Road-Prediction for Augmented Reality Navigation Systems at Night
Limmer, M., Forster, J., Baudach, D., Schüle, F., Schweiger, R., & Lensch, H. P. A. (2016) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2016.7795862 · Cited by 31
Diagnosis support of sickle cell anemia by classifying red blood cell shape in peripheral blood images
Delgado-Font, W., Escobedo-Nicot, M., González-Hidalgo, M., Herold-García, S., Jaume-i-Capó, A., & Mir, A. (2020) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-019-02085-9 · Cited by 29
Interpretable Machine Learning Assessment
Han, H., Wu, Y., Wang, J., & Han, A. (2022) · Neurocomputing · doi:10.1016/j.neucom.2023.126891 · Cited by 31
Diagnosis system for cancer disease using a single setting approach
Bhuyan, H. K., Vijayaraj, A., & Ravi, V. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15478-8 · Cited by 28

Authors: Jurman, G., Riccadonna, S., & Furlanello, C.
Affiliations: Fondazione Bruno Kessler (IT)
Corresponding author: Jurman, Giuseppe
ORCID: Jurman, Giuseppe
Riccadonna, Samantha
Furlanello, Cesare
Venue: PLoS ONE
Publisher: Public Library of Science (PLoS)
Published: 2012-08-08
Volume: 7
Issue: 8
Pages: e41882
ISSN: 1932-6203 (ISSN-L)
DOI: 10.1371/journal.pone.0041882
PMC: PMC3414515
OpenAlex: W1988301066
Cited by: 406
References: 43
Citations by year: 2026: 17 · 2025: 29 · 2024: 38 · 2023: 54 · 2022: 47 · 2021: 54 · 2020: 32 · 2019: 36 · 2018: 25 · 2017: 23 · 2016: 15 · 2015: 12 · 2014: 14 · 2013: 10
FWCI: 11.34
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
creativecommons.org/licenses/by/4.0/ (unspecified)
Locations: PLoS ONE (publishedVersion, cc-by)
Europe PMC (PubMed Central) (publishedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
arXiv (Cornell University) (submittedVersion)
RePEc: Research Papers in Economics (submittedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders: European Commission
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
Keywords: Matthews correlation coefficient, multi-class classification, generalization, binary classification, monotonicity
MeSH terms: Algorithms, Artificial Intelligence, Computers, Genotype, Probability, ROC Curve, Software, Statistics as Topic, Reproducibility of Results, Models, Statistical, Entropy, Computational Biology, Area Under Curve, Oligonucleotide Array Sequence Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Comparison of MCC and CEN Error Measures in Multi-Class Prediction

Abstract

Overview

Integrity signals

Figures

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks