Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Data imbalance is frequently encountered in biomedical applications. Resampling techniques can be used in binary classification to tackle this issue. However such solutions are not desired when the number of samples in the small class is limited. Moreover the use of inadequate performance metrics, such as accuracy, lead to poor generalization results because the classifiers tend to predict the largest size class. One of the good approaches to deal with this issue is to optimize performance metrics that are designed to handle data imbalance. Matthews Correlation Coefficient (MCC) is widely used in Bioinformatics as a performance metric. We are interested in developing a new classifier based on the MCC metric to handle imbalanced data. We derive an optimal Bayes classifier for the MCC metric using an approach based on Frechet derivative. We show that the proposed algorithm has the nice theoretical property of consistency. Using simulated data, we verify the correctness of our optimality result by searching in the space of all possible binary classifiers. The proposed classifier is evaluated on 64 datasets from a wide range data imbalance. We compare both classification performance and CPU efficiency for three classifiers: 1) the proposed algorithm (MCC-classifier), the Bayes classifier with a default threshold (MCC-base) and imbalanced SVM (SVM-imba). The experimental evaluation shows that MCC-classifier has a close performance to SVM-imba while being simpler and more efficient.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“since it has been shown to present a balanced score when dealing with imbalanced data [11, 6] as is our case with fewer manipulated images.”
From BioFors: A Large Biomedical Image Forensics Dataset (2021) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399
A Probabilistic Theory of Pattern Recognition
Devroye, L., Györfi, L., & Lugosi, G. (1996) · Stochastic Modelling and Applied Probability · doi:10.1007/978-1-4612-0711-5 · Cited by 3,274
Assessing the accuracy of prediction algorithms for classification: an overview
Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A. F., & Nielsen, H. (2000) · Bioinformatics · doi:10.1093/bioinformatics/16.5.412 · Cited by 2,111
A sequential algorithm for training text classifiers
Lewis, D., & Gale, W. A. (1994) · arXiv (Cornell University) · doi:10.5555/188490.188495 · Cited by 1,731
A Sequential Algorithm for Training Text Classifiers
Lewis, D. D., & Gale, W. A. (1994) · SIGIR ’94 · doi:10.1007/978-1-4471-2099-5_1 · Cited by 1,208
Applying Support Vector Machines to Imbalanced Datasets
Akbani, R., Kwek, S., & Japkowicz, N. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30115-8_7 · Cited by 1,134
Yang, J., Roy, A., & Zhang, Y. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt447 · Cited by 1,035
MAQC Consortium (2010) · Nature Biotechnology · doi:10.1038/nbt.1665 · Cited by 889
Imbalanced Learning: Foundations, Algorithms, and Applications
(2013) · Wiley eBooks · doi:10.1002/9781118646106 · Cited by 831
Cited by
Chicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143
Classification assessment methods
Tharwat, A. (2021) · Applied Computing and Informatics · doi:10.1016/j.aci.2018.08.003 · Cited by 2,417
“A coefficient of þ 1 indicates a perfect prediction, − 1 represents total disagreement between prediction and true values and zero means that no better than random prediction [16,3].”
BackgroundEvaluation metrics and statistical tests for machine learning
Rainio, O., Teuho, J., & Klén, R. (2024) · Scientific Reports · doi:10.1038/s41598-024-56706-x · Cited by 1,338
“· = = · = = Similarly, MCC can be computed from a general k × k confusion matrix with the formula 27 In the special case k = 2 , we obtain the same formulas for Cohen’s κ and MCC as in (2) and (3) 22 .”
MethodLuque, A., Carrasco, A., Martín, A., & de las Heras, A. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.02.023 · Cited by 1,128
On evaluation metrics for medical applications of artificial intelligence
Hicks, S. A., Strümke, I., Thambawita, V., Hammou, M., Riegler, M. A., Halvorsen, P., & Parasa, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-09954-8 · Cited by 934
Chicco, D., Tötsch, N., & Jurman, G. (2021) · BioData Mining · doi:10.1186/s13040-021-00244-z · Cited by 893
Metrics for Multi-Class Classification: an OverviewPreprint
Grandini, M., Bagli, E., & Visani, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.05756 · Cited by 2,153
“Macro-Average methods tend to calculate an overall mean of different measures, because the numerators of Macro Average Precision and Macro Average Recall are composed by values in the range [0, 1].”
MethodRifaioglu, A. S., Atas, H., Martin, M. J., Cetin-Atalay, R., Atalay, V., & Doğan, T. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bby061 · Cited by 529
Naser, M. Z., & Alavi, A. H. (2023) · Architecture, Structures and Construction · doi:10.1007/s44150-021-00015-8 · Cited by 464
“• Considered as a balanced measures as it involves values of all the four quardants of a confusion matrix [100].”
BackgroundChicco, D., Warrens, M. J., & Jurman, G. (2021) · IEEE Access · doi:10.1109/access.2021.3084050 · Cited by 444
“Recently, Boughorbel and colleagues [13] described an enhanced classifier based on the Matthews correlation coefficient, while Zhu [14] investigated the behavior of MCC on several imbalanced cases.”
BackgroundPredicting long-term dynamics of soil salinity and sodicity on a global scale
Hassani, A., Azapagic, A., & Shokri, N. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2013771117 · Cited by 440
Baryannis, G., Dani, S., & Antoniou, G. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.07.059 · Cited by 333
“date that was accepted by the tier 1 supplier is not in the last 9 weeks of the year (left branch of X[31] ≤ 0.”
BackgroundA Neural Network Ensemble With Feature Engineering for Improved Credit Card Fraud Detection
Esenogho, E., Mienye, I. D., Swart, T. G., Aruleba, K., & Obaido, G. (2022) · IEEE Access · doi:10.1109/access.2022.3148298 · Cited by 332
“Data-level techniques tend to create a balanced dataset by undersampling the majority class or oversampling the minority class, sometimes the combination of both [11].”
BackgroundThorsen-Meyer, H.-C., Nielsen, A. B., Nielsen, A. P., Kaas-Hansen, B. S., Toft, P., Schierbeck, J., Strøm, T., Chmura, P. J., Heimann, M., Dybdahl, L., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30018-2 · Cited by 336
Lenselink, E. B., ten Dijke, N., Bongers, B., Papadatos, G., van Vlijmen, H. W. T., Kowalczyk, W., IJzerman, A. P., & van Westen, G. J. P. (2017) · Journal of Cheminformatics · doi:10.1186/s13321-017-0232-0 · Cited by 334
“The MCC was calculated to represent the global model quality, and has been shown to be a good metric for unbalanced data-sets [25].”
MethodWaldner, F., & Diakogiannis, F. I. (2020) · Remote Sensing of Environment · doi:10.1016/j.rse.2020.111741 · Cited by 310
Foreground segmentation using convolutional neural networks for multiscale feature encoding
Lim, L. A., & Yalim Keles, H. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.08.002 · Cited by 305
“However, as it is also claimed by [35], F-Measure is sensitive to imbalanced data since it does not incorporate true negatives into account.”
BackgroundMason, D. M., Friedensohn, S., Weber, C. R., Jordi, C., Wagner, B., Meng, S. M., Ehling, R. A., Bonati, L., Dahinden, J., Gainza, P., et al. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-021-00699-9 · Cited by 308
Software Defect Prediction Using Ensemble Learning: A Systematic Literature Review
Matloob, F., Ghazal, T. M., Taleb, N., Aftab, S., Ahmad, M., Khan, M. A., Abbas, S., & Soomro, T. R. (2021) · IEEE Access · doi:10.1109/access.2021.3095559 · Cited by 293
“account all true and false positives and negatives to determine the quality of binary classification [14].”
BackgroundElmasry, W., Akbulut, A., & Zaim, A. H. (2020) · Computer Networks · doi:10.1016/j.comnet.2019.107042 · Cited by 277
A systematic evaluation of single cell RNA-seq analysis pipelines
Vieth, B., Parekh, S., Ziegenhain, C., Enard, W., & Hellmann, I. (2019) · Nature Communications · doi:10.1038/s41467-019-12266-7 · Cited by 265
“To summarise the whole confusion matrix in one representative value we chose the Matthews Correlation Coefficient ( MCC = ), because it is a balanced measure ensuring a reliable comparison of method performance across all DE-settings 45,46 .”
MethodZhou, X., Lu, P., Zheng, Z., Tolliver, D., & Keramati, A. (2020) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2020.106931 · Cited by 255
Ileberi, E., Sun, Y., & Wang, Z. (2021) · IEEE Access · doi:10.1109/access.2021.3134330 · Cited by 244
“In this research, we consider the Matthews correlation coefficient (MCC) [31], [32], the AUC [33], and the Confusion Matrix (CM) as additional performance indicators.”
MethodAccelerometer-Based Human Fall Detection Using Convolutional Neural Networks
Santos, G. L., Endo, P. T., Monteiro, K. H. D. C., Rocha, E. D. S., Silva, I., & Lynn, T. (2019) · Sensors · doi:10.3390/s19071644 · Cited by 232
“MCC is a metric used widely to measure performance when dealing with imbalanced data and is defined in terms of TP, TN, FP and FN [22]:”
BackgroundSelf-paced Ensemble for Highly Imbalanced Massive Data Classification
Liu, Z., Cao, W., Gao, Z., Bian, J., Chen, H., Chang, Y., & Liu, T.-Y. (2020) · 2020 IEEE 36th International Conference on Data Engineering (ICDE) · doi:10.1109/icde48307.2020.00078 · Cited by 173
“, Matthews correlation coefficient) [24], and AUCPRC (i.”
BackgroundMachine learning classification can reduce false positives in structure-based virtual screening
Adeshina, Y. O., Deeds, E. J., & Karanicolas, J. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2000585117 · Cited by 223
“In addition to making compelling decoys, the proportion of decoys-to-actives also has a significant effect on the performance of machine learning trained model (55).”
AnOxPePred: using deep learning for the prediction of antioxidative properties of peptides
Olsen, T. H., Yesiltas, B., Marin, F. I., Pertseva, M., García-Moreno, P. J., Gregersen, S., Overgaard, M. T., Jacobsen, C., Lund, O., Hansen, E. B., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-78319-w · Cited by 215
(2020) · Progress in Molecular Biology and Translational Science · doi:10.1016/bs.pmbts.2020.04.003 · Cited by 212
Picek, S., Heuser, A., Jovic, A., Bhasin, S., & Regazzoni, F. (2018) · IACR Transactions on Cryptographic Hardware and Embedded Systems · doi:10.46586/tches.v2019.i1.209-237 · Cited by 208
Sedik, A., Iliyasu, A. M., Abd El-Rahiem, B., Abdel Samea, M. E., Abdel-Raheem, A., Hammad, M., Peng, J., Abd El-Samie, F. E., & Abd El-Latif, A. A. (2020) · Viruses · doi:10.3390/v12070769 · Cited by 203
“As deduced from that figure, both MCC and Accuracy (A) take into consideration the TP, FP, TN and FN, whereas MCC is generally regarded as a balanced measure that can be used even if classes are of different sizes [44], Accuracy takes into account the number of sets applied in the five-fold and the number of classes,…”
MethodOn the performance of Matthews correlation coefficient (MCC) for imbalanced dataset
Zhu, Q. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.03.030 · Cited by 202
Automatic COVID-19 lung infected region segmentation and measurement using CT-scans images
Oulefki, A., Agaian, S., Trongtirakul, T., & Kassah Laouar, A. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2020.107747 · Cited by 199
A survey on deep learning for skin lesion segmentation
Mirikharaji, Z., Abhishek, K., Bissoto, A., Barata, C., Avila, S., Valle, E., Celebi, M. E., & Hamarneh, G. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102863 · Cited by 189
“, 2020; Zhu, 2020), this measure appears to be suitable for class-imbalanced domains (Boughorbel et al., 2017; Chicco and Jurman, 2020; Luque et al., 2020).”
BackgroundCross-validation Metrics for Evaluating Classification Performance on Imbalanced Data
Wardhani, N. W. S., Rochayani, M. Y., Iriany, A., Sulistyono, A. D., & Lestantyo, P. (2019) · 2019 International Conference on Computer, Control, Informatics and its Applications (IC3INA) · doi:10.1109/ic3ina48034.2019.8949568 · Cited by 166
“Research [4] compared MCC, AUC, Accuracy, and F1 on imbalanced simulation data with three classifiers.”
MethodFoody, G. M. (2023) · PLOS ONE · doi:10.1371/journal.pone.0291908 · Cited by 186
“Although some claims to the MCC being robust or relatively unaffected by class imbalance have been made [52] it is known that the MCC can be impacted by prevalence.”
BackgroundOn the platform but will they buy? Predicting customers' purchase behavior using deep learning
Chaudhuri, N., Gupta, G., Vamsi, V., & Bose, I. (2021) · Decision Support Systems · doi:10.1016/j.dss.2021.113622 · Cited by 181
Li, Y., Zhang, M., & Chen, C. (2022) · Applied Energy · doi:10.1016/j.apenergy.2021.118347 · Cited by 184
“d) F1-score The F1-score is a weighted average of the precision and recall for a specified confidence threshold to estimate overall classification accuracy, which is given as follows [22, 30, 38].”
BackgroundMethodAmerican sign language recognition and training method with recurrent neural network
Lee, C. K. M., Ng, K. K. H., Chen, C.-H., Lau, H. C. W., Chung, S. Y., & Tsoi, T. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114403 · Cited by 178
Customer Emotions in Service Robot Encounters: A Hybrid Machine-Human Intelligence Approach
Filieri, R., Lin, Z., Li, Y., Lu, X., & Yang, X. (2022) · Journal of Service Research · doi:10.1177/10946705221103937 · Cited by 160
“…proposed emotion classi fi cation model by comparing its performance with other benchmark models using the Mathews Correlation Coef fi cient (MCC) metric (Boughorbel et al. 2017; Matthews 1975), which is calculated as where TP, TN, FP, and FN refer to the number of true positive, true negative,…”
MethodComputational Design of Stable and Soluble Biocatalysts
Musil, M., Konegger, H., Hon, J., Bednar, D., & Damborsky, J. (2019) · ACS Catalysis · doi:10.1021/acscatal.8b03613 · Cited by 160
Kouadri, A., Hajji, M., Harkat, M.-F., Abodayeh, K., Mansouri, M., Nounou, H., & Nounou, M. (2020) · Renewable Energy · doi:10.1016/j.renene.2020.01.010 · Cited by 159
Algorithm Supported Induction for Building Theory: How Can We Use Prediction Models to Theorize?
Shrestha, Y. R., He, V. F., Puranam, P., & von Krogh, G. (2021) · Organization Science · doi:10.1287/orsc.2020.1382 · Cited by 156
Analysis of Autoencoders for Network Intrusion Detection
Song, Y., Hyun, S., & Cheong, Y.-G. (2021) · Sensors · doi:10.3390/s21134294 · Cited by 151
“Hence, MCC can effectively represent the model’s performance even when there is class imbalance [52] and is used for measuring IDS classification performance [41].”
MethodA review on antimicrobial peptides databases and the computational tools
Ramazi, S., Mohammadi, N., Allahverdi, A., Khalili, E., & Abdolmaleki, P. (2022) · Database · doi:10.1093/database/baac011 · Cited by 176
“Matthews Correlation Coefficient (MCC):MCC shows the correlation between true and predicted labels (139).”
Chongomweru, H., Kasem, A., & Newaz, S. H. S. (2019) · International Conference on Machine Learning and Soft Computing (ICMLSC) · doi:10.1145/3310986.3311023 · Cited by 182
“Some researchers have formulated new classifiers based on MCC, for-example Boughorbel et al in [2] proposed a new optimal classier called “MCC-classifier” based on MCC metric for handling imbalanced data, they evaluated their proposed classier on 64 real world imbalanced datasets and empirically compared its performan…”
MethodA New Deep Hybrid Boosted and Ensemble Learning-Based Brain Tumor Analysis Using MRI
Zahoor, M. M., Qureshi, S. A., Bibi, S., Khan, S. H., Khan, A., Ghafoor, U., & Bhutta, M. R. (2022) · Sensors · doi:10.3390/s22072726 · Cited by 187
Categorizing the Content of GitHub README Files
Prana, G. A. A., Treude, C., Thung, F., Atapattu, T., & Lo, D. (2019) · Empirical Software Engineering · doi:10.1007/s10664-018-9660-3 · Cited by 143
DRTHIS: Deep ransomware threat hunting and intelligence system at the fog layer
Homayoun, S., Dehghantanha, A., Ahmadzadeh, M., Hashemi, S., Khayami, R., Choo, K.-K. R., & Newton, D. E. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2018.07.045 · Cited by 143
Machine learning-based EEG signals classification model for epileptic seizure detection
Aayesha, A., Qureshi, M. B., Afzaal, M., Qureshi, M. S., & Fayaz, M. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-10597-6 · Cited by 158
Thambawita, V. L., Strümke, I., Hicks, S. A., Halvorsen, P., Parasa, S., & Riegler, M. A. (2021) · Diagnostics · doi:10.3390/diagnostics11122183 · Cited by 171
“Thus, we report macro averaged results for these three metrics [14], but not for MCC since it is robust against bias in the classes [15].”
MethodGalanis, K. A., Nastou, K. C., Papandreou, N. C., Petichakis, G. N., Pigis, D. G., & Iconomidou, V. A. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms22063210 · Cited by 151
“Out of these metrics, significant attention was given to MCC, since it is generally regarded as the best performance metric for binary classifiers [70,71].”
BackgroundBalanced-MixUp for Highly Imbalanced Medical Image Classification
Galdrán, A., Carneiro, G., & Ballester, M. Á. G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87240-3_31 · Cited by 176
Comprehensive Immune Monitoring of Clinical Trials to Advance Human Immunotherapy
Hartmann, F. J., Babdor, J., Gherardini, P. F., Amir, E.-A. D., Jones, K., Sahaf, B., Marquez, D. M., Krutzik, P., O’Donnell, E., Sigal, N., et al. (2019) · Cell Reports · doi:10.1016/j.celrep.2019.06.049 · Cited by 124
Wang, M. W. H., Goodman, J. M., & Allen, T. E. H. (2020) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.0c00316 · Cited by 142
A one-class classification approach for bot detection on Twitter
Rodríguez‐Ruiz, J., Mata-Sánchez, J. I., Monroy, R., Loyola‐González, O., & López‐Cuevas, A. (2020) · Computers & Security · doi:10.1016/j.cose.2020.101715 · Cited by 148
Detecting Cryptomining Malware: a Deep Learning Approach for Static and Dynamic Analysis
Darabian, H., Homayoun, S., Dehghantanha, A., Hashemi, S., Karimipour, H., Parizi, R. M., & Choo, K. R. (2020) · Journal of Grid Computing · doi:10.1007/s10723-020-09510-6 · Cited by 122
Um, H., Tixier, F., Bermudez, D., Deasy, J. O., Young, R. J., & Veeraraghavan, H. (2019) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab2f44 · Cited by 120
Jiang, Y., Wang, D., Yao, Y., Eubel, H., Künzler, P., Møller, I. M., & Xu, D. H. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.08.027 · Cited by 134
“MCC considers true and false positives and negatives, and is generally regarded as a balanced measure even if the classes are of very different sizes [41].”
BackgroundSilva, J. C. F., Teixeira, R. M., Silva, F. F. E., Brommonschenkel, S. H., & Fontes, E. P. B. (2019) · Plant Science · doi:10.1016/j.plantsci.2019.03.020 · Cited by 126
Practices and Trends of Machine Learning Application in Nanotoxicology
Furxhi, I., Murphy, F., Mullins, M., Arvanitis, A., & Poland, C. A. (2020) · Nanomaterials · doi:10.3390/nano10010116 · Cited by 121
“MCC is equivalent to the Pearson Correlation Coefficient for binary variables [147] and has been selected as an evaluation metric for micro-array-based predictive models by the MicroArray Quality Control (MAQC) Consortium [148].”
MethodLaPierre, N., Ju, C. J.-T., Zhou, G., & Wang, W. (2019) · Methods · doi:10.1016/j.ymeth.2019.03.003 · Cited by 114
“Some papers also report the Matthews Correlation Coefficient (MCC) which is robust to case-control class imbalances [72].”
BackgroundSujon, K. M., hassan, R. B., Choi, K., & Samad, M. A. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01313-4 · Cited by 202
“Similarly, [4] advocated the use of MCC for binary classification but lacked a comparative evaluation of MCC against other metrics like F1-score and precision.”
Naser, M. Z. (2021) · Fire Technology · doi:10.1007/s10694-020-01069-8 · Cited by 125
Chicco, D., Старовойтов, В. В., & Jurman, G. (2021) · IEEE Access · doi:10.1109/access.2021.3068614 · Cited by 139
“[12] published a study based on the benefits of assessing the performances of several machine learningmethods on imbalanced datasets with MCC.”
BackgroundMachine learning in continuous casting of steel: a state-of-the-art survey
Cemernek, D., Cemernek, S., Gursch, H., Pandeshwar, A., Leitner, T., Berger, M., Klösch, G., & Kern, R. (2021) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-021-01754-7 · Cited by 116
“The MCC emerged in the field of biomedical research, since imbalance is a very common phenomenon (for example rare diseases) in this field (Boughorbel et al. 2017).”
BackgroundHyperspectral imaging for tissue classification, a way toward smart laparoscopic colorectal surgery
Baltussen, E. J. M., Kok, E. N. D., de Koning, S. G. B., Sanders, J., Aalbers, A. G. J., Kok, N. F. M., Beets, G. L., Flohil, C. C., Bruin, S. C., Kuhlmann, K. F. D., et al. (2019) · Journal of Biomedical Optics · doi:10.1117/1.jbo.24.1.016002 · Cited by 110
“Of these parameters, the MCC and AUC are most accurate in the current study, because these parameters are relatively insensitive to the effect of an imbalanced dataset.(27) However, for the combination of the quadratic classifier and the SVM, the MCC and AUC cannot be used, because both measures only account for a two…”
MethodComputational design of enzymes for biotechnological applications
Planas-Iglesias, J., Marques, S. M., Pinto, G., Musil, M., Štourač, J., Damborský, J., & Bednář, D. (2021) · Biotechnology Advances · doi:10.1016/j.biotechadv.2021.107696 · Cited by 115
Chandra, T. B., Verma, K., Singh, B. K., Jain, D., & Netam, S. S. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113514 · Cited by 113
Unsupervised Algorithms to Detect Zero-Day Attacks: Strategy and Application
Zoppi, T., Ceccarelli, A., & Bondavalli, A. (2021) · IEEE Access · doi:10.1109/access.2021.3090957 · Cited by 131
Lachance, J., Lloyd, C. J., Monk, J. M., Yang, L., Sastry, A. V., Seif, Y., Palsson, B. Ø., Rodrigue, S., Feist, A. M., King, Z. A., et al. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006971 · Cited by 125
“The Matthews Correlation Coefficient (MCC) is a metric frequently used to evaluate GEMs’ gene essentiality prediction, as it takes account of false and true positive and negative observations in a balanced way, and works with binary classifications [30].”
MethodDetect-and-segment: A deep learning approach to automate wound image segmentation
Scebba, G., Zhang, J., Catanzaro, S., Mihai, C., Distler, O., Berli, M., & Karlen, W. (2022) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2022.100884 · Cited by 134
Scalable and explainable legal prediction
Branting, L. K., Pfeifer, C., Brown, B., Ferro, L., Aberdeen, J., Weiss, B., Pfaff, M., & Liao, B. (2021) · Artificial Intelligence and Law · doi:10.1007/s10506-020-09273-1 · Cited by 70
“For the HAN, the Matthews Correlation Coefficient (MCC), a standard measure of the improvement of a binary classifier over a majority class rule that ranges from − 1 to + 1 (Boughorbel et al. 2017), was about 0.558.”
BackgroundMethodInfluentialEvaluation of single-cell classifiers for single-cell RNA sequencing data sets
Zhao, X., Wu, S., Fang, N., Sun, X., & Fan, J. (2020) · Briefings in Bioinformatics · doi:10.1093/bib/bbz096 · Cited by 97
“In addition, Matthews correlation coefficient (MCC) takes into account both true and false positives and negatives, which is regarded as a balanced measure for class-imbalanced data sets [41, 42] and defined as the following formula:”
MethodClinical data classification using an enhanced SMOTE and chaotic evolutionary feature selection
Sreejith, S., Nehemiah, H. K., & Kannan, A. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103991 · Cited by 118
Weighted kappa measures for ordinal multi-class classification performance
Yılmaz, A. E., & Demirhan, H. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110020 · Cited by 113
Machine Learning in Network Intrusion Detection: A Cross-Dataset Generalization Study
Cantone, M., Marrocco, C., & Bria, A. (2024) · IEEE Access · doi:10.1109/access.2024.3472907 · Cited by 182
“The MCC [67], [68], [69] is a correlation coefficient between classifier output and true label, hence it yields a more informative and truthful score in evaluating binary classifications than accuracy, especially in the case of unbalanced datasets.”
BackgroundPerformance Measures for Binary Classification
Berrar, D. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-809633-8.20351-8 · Cited by 133
A real-time electrical load forecasting and unsupervised anomaly detection framework
Wang, X., Yao, Z., & Papaefthymiou, M. C. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.120279 · Cited by 110
SNNRice6mA: A Deep Learning Method for Predicting DNA N6-Methyladenine Sites in Rice Genome
Yu, H., & Dai, Z. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.01071 · Cited by 99
“MCC is generally regarded as a balanced measure that can be used even if the samples are unbalanced in two classes (Boughorbel et al., 2017).”
BackgroundRanjbarzadeh, R., Zarbakhsh, P., Caputo, A., Tırkolaee, E. B., & Bendechache, M. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107723 · Cited by 126
Diallo, R., Edalo, C., & Awe, O. O. (2024) · STEAM-H: Science, Technology, Engineering, Agriculture, Mathematics & Health · doi:10.1007/978-3-031-72215-8_12 · Cited by 167
Structure–activity relationship-based chemical classification of highly imbalanced Tox21 datasets
Idakwo, G., Thangapandian, S., Luttrell, J., Li, Y., Wang, N., Zhou, Z., Hong, H., Yang, B., Zhang, C., & Gong, P. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-020-00468-x · Cited by 99
“Matthews correlation coefficient (MCC) offers a good index for the performance of imbalanced classification tasks as it incorporates all the components of the confusion matrix [53].”
Chakravarthy, S. R. S., & Rajaguru, H. (2019) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.22364 · Cited by 135
Leggieri, M. C., Mazzoni, M., Fodil, S., Moschini, M., Bertuzzi, T., Prandini, A., & Battilani, P. (2020) · Food Control · doi:10.1016/j.foodcont.2020.107722 · Cited by 92
Antimicrobial peptide identification using multi-scale convolutional network
Su, X. B., Xu, J., Yin, Y., Quan, X., & Zhang, H. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3327-y · Cited by 96
Aldegheishem, A., Anwar, M., Javaid, N., Alrajeh, N. A., Shafiq, M., & Ahmed, H. (2021) · IEEE Access · doi:10.1109/access.2021.3056566 · Cited by 98
“Matthews introduced Matthews correlation coefficient (MCC) [57], which is used as a quality measure in ML.”
MethodInfluentialKha, Q.-H., Ho, Q., & Le, N. Q. K. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c01034 · Cited by 90
“49,50 Six classifier algorithms, i.e., Random Forest (RF), Adaptive Boosting classifier (AB), Extra Tree Classifier (ET), Logistic regression (LR), Multilayer perceptron classifier (MLP), and eXtreme Gradient Boosting (XGB), were selected to make the performance comparison with our proposed model.”
MethodDeep learning GAN-based data generation and fault diagnosis in the data center HVAC system
Du, Z., Chen, K., Chen, S., He, J., Zhu, X., & Jin, X. (2023) · Energy and Buildings · doi:10.1016/j.enbuild.2023.113072 · Cited by 92
Jakobsen, P., Garcia-Ceja, E., Riegler, M. A., Stabell, L. A., Nordgreen, T., Tørresen, J., Fasmer, O. B., & Oedegaard, K. J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231995 · Cited by 89
Detecting Technical Anomalies in High-Frequency Water-Quality Data Using Artificial Neural Networks
Rodríguez‐Pérez, J., Leigh, C., Liquet, B., Kermorvant, C., Peterson, E. E., Sous, D., & Mengersen, K. L. (2020) · Environmental Science & Technology · doi:10.1021/acs.est.0c04069 · Cited by 91
A Large Scale Study of Long-Time Contributor Prediction for GitHub Projects
Bao, L., Xia, X., Lo, D., & Murphy, G. C. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2918536 · Cited by 103
“balanced measure which can be used even if the classes are highly imbalanced [7].”
BackgroundZoppi, T., Ceccarelli, A., Puccetti, T., & Bondavalli, A. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103107 · Cited by 104
Classifiers and their Metrics Quantified
Brown, J. B. (2018) · Molecular Informatics · doi:10.1002/minf.201700127 · Cited by 82
Kalid, S. N., Ng, K.-H., Tong, G.-K., & Khor, K. (2020) · IEEE Access · doi:10.1109/access.2020.2972009 · Cited by 95
“According to [27], general learning algorithms are able to give high accuracies on balanced data sets, but not on unbalanced data sets.”
BackgroundData augmentation in dermatology image recognition using machine learning
Aggarwal, S. L. P. (2019) · Skin Research and Technology · doi:10.1111/srt.12726 · Cited by 80
Shrestha, S., Zhang, C., Jerde, C. R., Nie, Q., Li, H., Offer, S. M., & Diasio, R. B. (2018) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1020 · Cited by 80
“MCC is one of the elective metrics in the US Food and Drug Administration (FDA)-led MicroArray Quality Control (MAQC-II) initiative, which compares and validates predictive models in individualized medicine.(39) To further optimize the predictive ability of each tool, a new prediction score threshold was selected that…”
BackgroundBroadband hyperspectral imaging for breast tumor detection using spectral and spatial information.
Kho, E., Dashtbozorg, B., de Boer, L. D., van de Vijver, K. K., Sterenborg, H., & Ruers, T. (2019) · Biomedical Optics Express · doi:10.1364/boe.10.004496 · Cited by 63
Gadaleta, D., Manganelli, S., Roncaglioni, A., Toma, C., Benfenati, E., & Mombelli, E. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00297 · Cited by 81
Hamuli, P. C., Sacré, P., Tchounga, C. A. W., Coïc, L., Avohou, H. T., Mbinze, J. K., Ngono, R. A. N., Marini, R. D., Hubert, P., & Ziémons, E. (2019) · Talanta · doi:10.1016/j.talanta.2019.04.049 · Cited by 83
“The main advantage of Matthews correlation coefficient over accuracy or F1 score is that it takes into account the class balance ratio and is less affected by unbalanced classes [45].”
BackgroundPérez, E., Reyes, Ó., & Ventura, S. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101858 · Cited by 85
Metadata
- Authors
- Boughorbel, S., Jarray, F., & El-Anbari, M.
- Affiliations
- Sidra Medical and Research Center (QA)
- Conservatoire National des Arts et Métiers (FR)
- Corresponding author
- Boughorbel, Sabri
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2017-06-02
- Volume
- 12
- Issue
- 6
- Pages
- e0177678
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5456046
- OpenAlex
- W2620760558
- Cited by
- 1,222
- References
- 29
- Citations by year
- 2026: 90 · 2025: 104 · 2024: 111 · 2023: 163 · 2022: 179 · 2021: 224 · 2020: 165 · 2019: 120 · 2018: 64 · 2017: 2
- FWCI
- 57.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Qatar Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Keywords
- class imbalance, Matthews correlation coefficient, Fréchet derivative, binary classification, biomedical classification
- MeSH terms
- Algorithms, Bayes Theorem, Models, Theoretical, Computational Biology, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.