Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Data imbalance is frequently encountered in biomedical applications. Resampling techniques can be used in binary classification to tackle this issue. However such solutions are not desired when the number of samples in the small class is limited. Moreover the use of inadequate performance metrics, such as accuracy, lead to poor generalization results because the classifiers tend to predict the largest size class. One of the good approaches to deal with this issue is to optimize performance metrics that are designed to handle data imbalance. Matthews Correlation Coefficient (MCC) is widely used in Bioinformatics as a performance metric. We are interested in developing a new classifier based on the MCC metric to handle imbalanced data. We derive an optimal Bayes classifier for the MCC metric using an approach based on Frechet derivative. We show that the proposed algorithm has the nice theoretical property of consistency. Using simulated data, we verify the correctness of our optimality result by searching in the space of all possible binary classifiers. The proposed classifier is evaluated on 64 datasets from a wide range data imbalance. We compare both classification performance and CPU efficiency for three classifiers: 1) the proposed algorithm (MCC-classifier), the Bayes classifier with a default threshold (MCC-base) and imbalanced SVM (SVM-imba). The experimental evaluation shows that MCC-classifier has a close performance to SVM-imba while being simpler and more efficient.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“since it has been shown to present a balanced score when dealing with imbalanced data [11, 6] as is our case with fewer manipulated images.”
From BioFors: A Large Biomedical Image Forensics Dataset (2021) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399
A Probabilistic Theory of Pattern Recognition
Devroye, L., Györfi, L., & Lugosi, G. (1996) · Stochastic Modelling and Applied Probability · doi:10.1007/978-1-4612-0711-5 · Cited by 3,274
Assessing the accuracy of prediction algorithms for classification: an overview
Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A. F., & Nielsen, H. (2000) · Bioinformatics · doi:10.1093/bioinformatics/16.5.412 · Cited by 2,111
A sequential algorithm for training text classifiers
Lewis, D., & Gale, W. A. (1994) · arXiv (Cornell University) · doi:10.5555/188490.188495 · Cited by 1,731
A Sequential Algorithm for Training Text Classifiers
Lewis, D. D., & Gale, W. A. (1994) · SIGIR ’94 · doi:10.1007/978-1-4471-2099-5_1 · Cited by 1,208
Applying Support Vector Machines to Imbalanced Datasets
Akbani, R., Kwek, S., & Japkowicz, N. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30115-8_7 · Cited by 1,134
Yang, J., Roy, A., & Zhang, Y. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt447 · Cited by 1,035
MAQC Consortium (2010) · Nature Biotechnology · doi:10.1038/nbt.1665 · Cited by 889
Imbalanced Learning: Foundations, Algorithms, and Applications
(2013) · Wiley eBooks · doi:10.1002/9781118646106 · Cited by 831
Cited by
Items 101–200 of 1,000
Rishikeshan, C. A., & Ramesh, H. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.08.014 · Cited by 79
AntiVPP 1.0: A portable tool for prediction of antiviral peptides
Lissabet, J. F. B., Belén, L. H., & Farías, J. G. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.02.011 · Cited by 79
“Its value ranges from −1 to 1 and a larger MCC means a better prediction [22].”
Huang, W.-C., Lee, P., Liu, Y., Chiang, A. A.-P., & Lai, F. (2020) · SLEEP · doi:10.1093/sleep/zsz295 · Cited by 77
“To optimize the discriminative ability using the fewest features, continuous and categorical features were selected by single-feature SVM Area under the Receiver Operating Characteristic (AUROC) and Matthews Correlation Coefficient (MCC), respectively, during the feature selection [22].”
MethodBanzato, T., Bernardini, M., Cherubini, G. B., & Zotti, A. (2018) · BMC Veterinary Research · doi:10.1186/s12917-018-1638-2 · Cited by 79
Development of a novel prognostic model to predict 6-month swallowing recovery after ischemic stroke
Lee, W. H., Lim, M. H., Seo, H. G., Seong, M. Y., Oh, B.-M., & Kim, S. (2020) · Stroke · doi:10.1161/strokeaha.119.027439 · Cited by 72
Gobert, C., Kudzal, A., Sietins, J. M., Mock, C., Sun, J., & McWilliams, B. (2020) · Additive manufacturing · doi:10.1016/j.addma.2020.101460 · Cited by 94
Mapping Dominant Tree Species of German Forests
Welle, T., Aschenbrenner, L., Kuonath, K., Kirmaier, S., & Franke, J. (2022) · Remote Sensing · doi:10.3390/rs14143330 · Cited by 72
Tripathy, R. K., Paternina, M. R. A., Arrieta, J. G., Zamora‐Mendez, A., & Naik, G. R. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.03.008 · Cited by 74
Long, C., Jo, E., & Nam, Y. (2021) · The Journal of Supercomputing · doi:10.1007/s11227-021-04076-w · Cited by 87
A Machine Learning Approach for the Classification of Kidney Cancer Subtypes Using miRNA Genome Data
Muhamed Ali, A., Zhuang, H., Ibrahim, A., Rehman, O., Huang, M., & Wu, A. (2018) · Applied Sciences · doi:10.3390/app8122422 · Cited by 70
Real-time Fault Detection on Small Fixed-Wing UAVs using Machine Learning
Bronz, M., Başkaya, E., Delahaye, D., & Puechmore, S. (2020) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc50938.2020.9256800 · Cited by 70
“• Matthews Correlation Coefficient takes into account true and false positives and negatives and is generally regarded as a balanced measure which can be used even if the classes are of very different sizes [23].”
Backgroundda Silveira, A. C. M., Sobrinho, Á., da Silva, L. D., Costa, E. D. B., Perkusich, A., & Pinheiro, M. E. (2022) · Applied Sciences · doi:10.3390/app12073673 · Cited by 87
“The MCC is used to assess the quality of ratings and is highly recommended for imbalanced data [40], given by the following equation:”
MethodAdversarial Learning of Privacy-Preserving and Task-Oriented Representations
Xiao, T., Tsai, Y., Sohn, K., Chandraker, M. K., & Yang, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6930 · Cited by 94
Li, H., Wang, Z., Guan, Z., Miao, J., Li, W., Yu, P., & Molina-Jiménez, C. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108080 · Cited by 70
Laengsri, V., Shoombuatong, W., Adirojananon, W., Nantasenamat, C., Prachayasittikul, V., & Nuchnoi, P. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0929-2 · Cited by 66
“These results indicated that SVM model can tackle the imbalanced dataset problem and provide desirable prediction results [48].”
Snyder, G. F., Rodriguez-Gomez, V., Lotz, J. M., Torrey, P., Quirk, A. C. N., Hernquist, L., Vogelsberger, M., & Freeman, P. E. (2019) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stz1059 · Cited by 65
“excellent measure of the quality of a binary classification, performing well even when the two classes have very different sizes (Boughorbel et al. 2017), an important property when mergers are rare.”
BackgroundMost Relevant Spectral Bands Identification for Brain Cancer Detection Using Hyperspectral Imaging
Martinez-Vega, B., León, R., Fabelo, H. A., Ortega, S., Piñeiro, J. F., Szołna, A., Hernandez, M., Espino, C., Jiménez-O’Shanahan, A., Carrera, D., et al. (2019) · Sensors · doi:10.3390/s19245481 · Cited by 65
Quah, T. K. N., Tay, Y. W. D., Lim, J. H., Tan, M. J., Wong, T. N., & Li, K. H. H. (2023) · Mathematics · doi:10.3390/math11061499 · Cited by 81
Darmawahyuni, A., Nurmaini, S., Sukemi, S., Caesarendra, W., Bhayyu, V., Rachmatullah, M. N., & Firdaus, F. (2019) · Algorithms · doi:10.3390/a12060118 · Cited by 70
“Therefore, this paper uses metric performance balanced accuracy (BACC) and Matthew’s Correlation Coefficient (MCC) to produce better analysis in imbalanced data of MI [26].”
BackgroundMethodInfluentialA U-Net-Based Approach for Tool Wear Area Detection and Identification
Miao, H., Zhao, Z., Sun, C., Li, B., & Yan, R. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3033457 · Cited by 79
“This loss function in (8) was first incorporated into the objective function in [27]”
BackgroundMachine Learning for Predicting Mycotoxin Occurrence in Maize
Leggieri, M. C., Mazzoni, M., & Battilani, P. (2021) · Frontiers in Microbiology · doi:10.3389/fmicb.2021.661132 · Cited by 76
Bomane, A., Gonçalvès, A., & Ballester, P. J. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.01041 · Cited by 66
“These counts are used for calculating classification metrics that summarize the predictive performance reached by a classifier: precision, recall, F1-scores (Van Rijsbergen, 1979), and Matthews Correlation Coefficient (MCC) (Matthews, 1975; Boughorbel et al., 2017).”
MethodSoftware defect prediction based on correlation weighted class association rule mining
Shao, Y., Liu, B., Wang, S., & Li, G. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105742 · Cited by 70
Dilated Deep Neural Network for Segmentation of Retinal Blood Vessels in Fundus Images
Biswas, R., Vasan, A., & Roy, S. S. (2019) · Iranian Journal of Science and Technology Transactions of Electrical Engineering · doi:10.1007/s40998-019-00213-7 · Cited by 62
Mourad, M. W., Moubayed, S. P., Dezube, A. R., Mourad, Y., Park, K. T., Torreblanca-Zanca, A., Torrecilla, J. S., Cancilla, J. C., & Wang, J. (2020) · Scientific Reports · doi:10.1038/s41598-020-62023-w · Cited by 77
“…values ranging from − 1, absolute disagreement between prediction and real observation, to + 1, perfect prediction, where 0 means random prediction 19 ), precision or positive predictive value (PPV), negative predictive value (NPV), and F1 score (harmonic mean of PPV and sensitivity) are also…”
BackgroundLong-term implicit memory for sequential auditory patterns in humans
Bianco, R., Harrison, P. M. C., Hu, M., Bolger, C., Picken, S., Pearce, M. T., & Chait, M. (2020) · eLife · doi:10.7554/elife.56073 · Cited by 76
“Instead, we computed the MCC score, which is a measure of the quality of a binary classification, applicable even when classes are of different sizes (Boughorbel et al., 2017; Powers, 2007).”
A review of big data analysis methods for baleen whale passive acoustic monitoring
Kowarski, K. A., & Moors-Murphy, H. B. (2020) · Marine Mammal Science · doi:10.1111/mms.12758 · Cited by 61
“imbalanced data often seen in PAM (Boughorbel et al., 2017).”
BackgroundInferring Gene Regulatory Networks Using the Improved Markov Blanket Discovery Algorithm
Liu, W., Jiang, Y., Li, P., Sun, X., Gan, W., Zhao, Q., & Tang, H. (2021) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-021-00478-9 · Cited by 68
Padash, S., Mohebbian, M. R., Adams, S. J., Henderson, R. D. E., & Babyn, P. (2022) · Pediatric Radiology · doi:10.1007/s00247-022-05368-w · Cited by 70
“Furthermore, according to the prevalence of abnormalities, some indices like AUC, F1-score and Matthews correlation coefficient (MCC) [95] should be reported in addition to accuracy but are often not reported.”
BackgroundAn Empirical Study of the Impact of Data Splitting Decisions on the Performance of AIOps Solutions
Lyu, Y., Li, H., Sayagh, M., Jiang, Z. M. (., & Hassan, A. E. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3447876 · Cited by 60
“The MCC considers true and false positives and negatives and is generally regarded as a balanced measure that can be used even if the classes are of different sizes [6], which is the case of often highly skewed operation data.”
BackgroundA Comparison of Techniques for Class Imbalance in Deep Learning Classification of Breast Cancer
Walsh, R., & Tardy, M. (2022) · Diagnostics · doi:10.3390/diagnostics13010067 · Cited by 79
“Several studies [46,48] have shown MCC to be an informative metric in the study of class imbalance problems.”
BackgroundEmpirical study on multiclass classification‐based network intrusion detection
Elmasry, W., Akbulut, A., & Zaim, A. H. (2019) · Computational Intelligence · doi:10.1111/coin.12220 · Cited by 74
“86 In addition, MCC has a range of − 1 to 1, where − 1 indicates a completely wrong classifier, whereas 1 indicates a completely correct classifier.”
BackgroundDeep Learning for SARS COV-2 Genome Sequences
Whata, A., & Chimedza, C. (2021) · IEEE Access · doi:10.1109/access.2021.3073728 · Cited by 62
The impact of context metrics on just-in-time defect prediction
Kondo, M., Germán, D. M., Mizuno, O., & Choi, E.-H. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09736-3 · Cited by 69
Deep neural networks understand investors better
Mahmoudi, N., Docherty, P., & Moscato, P. (2018) · Decision Support Systems · doi:10.1016/j.dss.2018.06.002 · Cited by 64
A Wearable Electrocardiogram Telemonitoring System for Atrial Fibrillation Detection
Shao, M., Zhou, Z., Bin, G., Bai, Y., & Wu, S. (2020) · Sensors · doi:10.3390/s20030606 · Cited by 69
“Matthews correlation coe ffi cient (MCC) [34] In this work, twenty-folds cross-validation was used during the training process.”
Fogarty, E. S., Swain, D. L., Cronin, G. M., Moraes, L. E., Bailey, D. W., & Trotter, M. G. (2021) · Animals · doi:10.3390/ani11020303 · Cited by 67
“MCC is widely used in bioinformatics for unbalanced classification [58], providing a score between −1 and 1, where 1 indicates prefect prediction, 0 indicates random prediction and −1 indicates total disagreement.”
MethodQSAR-Co-X: an open source toolkit for multitarget QSAR modelling
Halder, A. K., & Cordeiro, M. N. D. S. (2021) · Journal of Cheminformatics · doi:10.1186/s13321-021-00508-0 · Cited by 60
“7), as well as the area under the receiver operating characteristic curve (AUROC) [34–36].”
BackgroundInfluentialGrassland Mowing Detection Using Sentinel-1 Time Series: Potential and Limitations
Vroey, M. D., Radoux, J., & Defourny, P. (2021) · Remote Sensing · doi:10.3390/rs13030348 · Cited by 58
“The MCC was first introduced by Matthews [39] and the metric is particularly suited to measure the quality of an unbalanced binary classification [40].”
MethodLi, B., Shen, L., Zhao, Y., Yu, W. H., Lin, H., Chen, C., Li, Y., & Zeng, Q. (2023) · Journal of Colloid and Interface Science · doi:10.1016/j.jcis.2023.02.030 · Cited by 56
Lu, H., Gao, H., Ye, M., & Wang, X. (2019) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2019.2952102 · Cited by 62
“When FP and FN are all 0, the MCC is 1, indicating that the classification result is completely correct; when TP and TN are all 0, the MCC value is -1, indicating that the classification result is completely wrong.”
MethodInfluentialAsogbon, M. G., Samuel, O. W., Geng, Y., Oluwagbemi, O. O., Ji, N., Chen, S., Naik, G. R., Pang, F., & Li, G. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.105278 · Cited by 60
McCay, K. D., Hu, P., Shum, H. P. H., Woo, W. L., Marcroft, C., Embleton, N. D., Munteanu, A., & Ho, E. S. L. (2021) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2021.3138185 · Cited by 68
“as a consistent performance metric on imbalanced datasets [7], [12].”
BackgroundMahlknecht, J., Torres-Martínez, J. A., Kumar, M., Mora, A., Kaown, D., & Loge, F. J. (2023) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2023.166863 · Cited by 63
The AdaBoost Approach Tuned by Firefly Metaheuristics for Fraud Detection
Petrović, A., Bačanin, N., Zivkovic, M. Z., Marjanović, M., Antonijević, M., & Strumberger, I. (2022) · IEEE World Conference on Applied Intelligence and Computing (AIC) · doi:10.1109/aic55036.2022.9848902 · Cited by 54
“The added metrics in which the methods are compared in are the Confusion Matrix (CM), the AUC [51], and the MCC [52], [53].”
MethodDeep learning Using Physically-Informed Input Data for Wetland Identification
O’Neil, G. L., Goodall, J. L., Behl, M., & Saby, L. (2020) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104665 · Cited by 59
Baskozos, G., Themistocleous, A. C., Hébert, H. L., Pascal, M. M. V., John, J., Callaghan, B. C., Laycock, H. C., Granovsky, Y., Crombez, G., Yarnitsky, D., et al. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01890-x · Cited by 61
“This is a well-known issue in model benchmarking and validation [24, 64, 65] but is sometimes ignored in ML studies.”
Zarei, A., & Asl, B. M. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105626 · Cited by 56
Banzato, T., Causin, F., Puppa, A. D., Cester, G., Mazzai, L., & Zotti, A. (2019) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.26723 · Cited by 57
“values in the interval [–1, 1], with 1 showing a complete agreement, –1 a complete disagreement, and 0 showing that the prediction was uncorrelated with the ground truth.(20) A schematic representation of”
BackgroundHong, Y., Chen, Y., Shen, R., Chen, S., Xu, G., Cheng, H., Guo, L., Wei, Z., Yang, J., Liu, Y., et al. (2021) · Environmental Pollution · doi:10.1016/j.envpol.2021.118128 · Cited by 52
Fanizzi, A., Losurdo, L., Basile, T. M. A., Bellotti, R., Bottigli, U., Delogu, P., Diacono, D., Didonna, V., Fausto, A., Lombardi, A., et al. (2019) · Journal of Clinical Medicine · doi:10.3390/jcm8060891 · Cited by 59
“The MCC is often used in machine learning as a measurement of the quality of binary classification [49] to compare performance results when the classes are very different in sizes because it also considers both true and false positives and false negatives against other similar coefficients [50].”
MethodDevelopment of a Clinical Scoring System for Bovine Respiratory Disease in Weaned Dairy Calves
Maier, G. U., Rowe, J. D., Lehenbauer, T. W., Karle, B. M., Williams, D. R., Champagne, J. D., & Aly, S. S. (2019) · Journal of Dairy Science · doi:10.3168/jds.2018-15474 · Cited by 51
Real-Time Classification of Green Coffee Beans by Using a Convolutional Neural Network
Huang, N.-F., Chou, D., & Lee, C.-A. (2019) · International Conference on Imaging, Signal Processing and Communications (ICISPC) · doi:10.1109/icispc.2019.8935644 · Cited by 51
“The overall coffee bean identification accuracy was approximately 93.34% and the FPR was 0.1007.”
BackgroundInfluentialTissue classification of oncologic esophageal resectates based on hyperspectral data
Maktabi, M., Köhler, H., Ivanova, M., Jansen‐Winkeln, B., Takoh, J., Niebisch, S., Rabe, S. M., Neumuth, T., Gockel, I., & Chalopin, C. (2019) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-019-02016-x · Cited by 53
A Local Feature Engineering Strategy to Improve Network Anomaly Detection
Carta, S., Podda, A. S., Recupero, D. R., & Saia, R. (2020) · Future Internet · doi:10.3390/fi12100177 · Cited by 53
“MCC(Ê) = (TP · TN)− (FP · FN)√ (TP + FP) · (TP + FN) · (TN + FP) · (TN + FN) (7) • The formalization of the AUC metric, which is able to evaluate the capability of a classifier to differentiate the normal from the intrusion events, regardless their numerical imbalance, is shown in Equation (8).”
MethodInfluentialBenchMetrics: a systematic benchmarking method for binary classification performance metrics
Canbek, G., Temizel, T. T., & Sağıroğlu, Ş. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06103-6 · Cited by 65
Colon Tissues Classification and Localization in Whole Slide Images Using Deep Learning
Gupta, P., Huang, Y., Sahoo, P. K., You, J., Chiang, S.-F., Onthoni, D. D., Chern, Y., Chao, K., Chiang, J., Yeh, C.-Y., et al. (2021) · Diagnostics · doi:10.3390/diagnostics11081398 · Cited by 55
“Therefore, different performance metrics [24] such as recall (sensitivity), specificity, precision, accuracy, F-score, and Matthew correlation coefficient (MCC) [25] were considered to evaluate different CNNs.”
MethodCOVID-19 diagnosis from routine blood tests using artificial intelligence techniques
Rikan, S. B., Azar, A. S., Ghafari, A., Bagherzadeh, J., & Pirnejad, H. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.103263 · Cited by 58
“Then, to evaluate the performance of the proposed models, the values of accuracy, precision, recall or sensitivity, specificity, F1-Score, area under the curve (AUC), and Mat-thews correlation coefficient (MCC) [33 – 35] were calculated for each of them.”
Semantic Segmentation of Airborne LiDAR Data in Maya Archaeology
Bundzel, M., Jaščur, M., Kováč, M., Lieskovský, T., Sinčák, P., & Tkáčik, T. (2020) · Remote Sensing · doi:10.3390/rs12223685 · Cited by 50
Sugeno integral generalization applied to improve adaptive image binarization
Bardozzo, F., de la Osa, B., Horanská, Ľ., Fumanal-Idocin, J., Priscoli, M. D., Troiano, L., Tagliaferri, R., Fernández, J., & Bustince, H. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.10.020 · Cited by 52
A practical perspective on knowledge space theory: ALEKS and its data
Cosyn, E., Uzun, H., Doble, C. W., & Matayoshi, J. (2021) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2021.102512 · Cited by 58
“4We also evaluated the performance of the assessment using the Matthews correlation coefficient (Matthews, 1975), which is recommended as a good overall measure for evaluating binary classifiers (Boughorbel et al., 2017; Chicco, 2017; Powers, 2011).”
MethodFang, X. L., Deshmukh, M., Chee, M. L., Soh, Z. D., Teo, Z. L., Thakur, S., Goh, J. H. L., Liu, Y., Husain, R., Mehta, J. S., et al. (2021) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2021-318866 · Cited by 56
PlasForest: a homology-based random forest classifier for plasmid detection in genomic datasets
Pradier, L., Tissot, T., Fiston-Lavier, A.-S., & Bedhomme, S. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04270-w · Cited by 56
Talaei-Khoei, A., & Wilson, J. M. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.08.008 · Cited by 48
SUSM: a scenario-based urban growth simulation model using remote sensing data
Kantakumar, L. N., Kumar, S., & Schneider, K. (2019) · European Journal of Remote Sensing · doi:10.1080/22797254.2019.1585209 · Cited by 56
“Thus, SUSM uses a balanced measure namely the “Matthews correlation coefficient (MCC)” to assess the quality of model simulations (Boughorbel et al., 2017; Powers, 2011).”
MethodModelling the impact of climate change on Tanzanian forests
John, E., Bunting, P., Hardy, A., Roberts, O., Giliba, R. A., & Silayo, D. S. (2020) · Diversity and Distributions · doi:10.1111/ddi.13152 · Cited by 58
Meta-Learning to Improve Unsupervised Intrusion Detection in Cyber-Physical Systems
Zoppi, T., Gharib, M., Atif, M., & Bondavalli, A. (2021) · ACM Transactions on Cyber-Physical Systems · doi:10.1145/3467470 · Cited by 60
“To such an extent, this paper primarily focuses on MCC , which does not suffer from the weaknesses mentioned above [87], [88].”
BackgroundA space-embedding strategy for anomaly detection in multivariate time series
Ji, Z., Wang, Y., Yan, K., Xie, X., Xiang, Y., & Huang, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117892 · Cited by 49
Exemplar-Based Sparse Representations for Detection of Parkinson's Disease From Speech
Reddy, M. K., & Alku, P. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3260709 · Cited by 63
“The MCC metric is a contingency matrix method of calculating the Pearson product-moment correlation coefficient [53] between actual and predicted values.”
BackgroundMethodMultimodal Deep Convolutional Neural Network Pipeline for AI-Assisted Early Detection of Oral Cancer
Devindi, G. A. I., Dissanayake, D. M. D. R., Liyanage, S., Francis, F. B. A. H., Pavithya, M. B. D., Piyarathne, N. S., Hettiarachchi, P. V. K. S., Rasnayaka, R. M. S. G. K., Jayasinghe, R. D., Ragel, R., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3454338 · Cited by 54
“Matthews Correlation Coefficient (MCC) considers all aspects of the confusion matrix and is often preferred for binary classification tasks due to its balanced assessment of true and false positives/negatives and its trustworthiness [47], [48].”
BackgroundInfluentialA 2.2- $\mu$ W Cognitive Always-On Wake-Up Circuit for Event-Driven Duty-Cycling of IoT Sensor Nodes
Rovere, G., Fateh, S., & Benini, L. (2018) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2018.2828505 · Cited by 49
“is adopted to allow classifier performance comparison in presence of unbalanced-class dataset [23].”
MethodA Feed-Forward and Pattern Recognition ANN Model for Network Intrusion Detection
Iqbal, A., & Aftab, S. (2019) · International Journal of Computer Network and Information Security · doi:10.5815/ijcnis.2019.04.03 · Cited by 47
“It is defined as the ratio between the observed and predicted binary classifications [39].”
BackgroundUnsupervised Anomaly Detectors to Detect Intrusions in the Current Threat Landscape
Zoppi, T., Ceccarelli, A., Capecchi, T., & Bondavalli, A. (2021) · ACM/IMS Transactions on Data Science · doi:10.1145/3441140 · Cited by 65
“As in Reference [22], aggregated metrics such as Precision, Recall (or Coverage), False Positive Rate, Accuracy, F-Score(β), F-Measure (F1), Area Under ROC Curve (AUC) [24], and Matthews Coefficient (MCC) [23] are used in different studies, depending on the domain.”
MethodNon-destructive Near Infrared Spectroscopy for the labelling of frozen Iberian pork loins
Cáceres-Nevado, J. M., Garrido‐Varo, A., Sanz, E. J. D. P., Tejerina, D., & Pérez‐Marín, D. (2021) · Meat Science · doi:10.1016/j.meatsci.2021.108440 · Cited by 47
Abhishek, K., & Hamarneh, G. (2021) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi48211.2021.9433782 · Cited by 49
“MCC is an informative metric even when dealing with skewed distributions [15] and has been shown to be an optimal metric when designing classifiers for imbalanced classes [16].”
BackgroundA Hybrid Unsupervised Approach for Retinal Vessel Segmentation
Khan, K. B., Siddique, M. S., Ahmad, M. I., & Mazzara, M. (2020) · BioMed Research International · doi:10.1155/2020/8365783 · Cited by 47
“The MCC and CAL values of Chauduri et al. [31], Niemeijer et al. [11], Hoover et al. [53], and B-COSFIRE [5] are calculated by utilizing their publicly accessible segmented images.”
BackgroundInfluentialVowels, L. M., Vowels, M. J., & Mark, K. P. (2021) · The Journal of Sex Research · doi:10.1080/00224499.2021.1967846 · Cited by 69
“The MCC coefficient can be interpreted as an overall effect size for the model using established effect size guidelines for Pearson’s correlation: .1 = small, .3 = medium, and .5 = large effect (Cohen, 1992).”
BackgroundInfluentialLearning From High-Dimensional Biomedical Datasets: The Issue of Class Imbalance
Pes, B. (2020) · IEEE Access · doi:10.1109/access.2020.2966296 · Cited by 54
“theMatthews Correlation (MC) coefficient takes into account the balance ratios of the four confusionmatrix categories (TP, TN, FP, FN), and it is considered an informative score to establish the quality of a binary classifier, even in a class imbalanced scenario [44], [45].”
BackgroundMachine Learning Techniques for Soybean Charcoal Rot Disease Prediction
Khalili, E., Kouchaki, S., Ramazi, S., & Ghanati, F. (2020) · Frontiers in Plant Science · doi:10.3389/fpls.2020.590529 · Cited by 57
Evolving data-adaptive support vector machines for binary classification
Dudzik, W., Nalepa, J., & Kawulok, M. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107221 · Cited by 49
Parasa, S., Repici, A., Berzin, T. M., Leggett, C. L., Gross, S. A., & Sharma, P. (2023) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2022.10.016 · Cited by 55
Bordner, A., Aouad, T., Medina, C. L., Yang, S., Moltó, A., Talbot, H., Dougados, M., & Feydy, A. (2023) · Diagnostic and Interventional Imaging · doi:10.1016/j.diii.2023.03.008 · Cited by 46
Identification of active molecules againstMycobacterium tuberculosisthrough machine learning
Ye, Q., Chai, X., Jiang, D., Liu, Y., Shen, C., Zhang, X., Li, D., Cao, D., & Hou, T. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab068 · Cited by 60
Koblitz, J., Reimer, L. C., Pukall, R., & Overmann, J. (2025) · Communications Biology · doi:10.1038/s42003-025-08313-3 · Cited by 56
“The receiver operating characteristic (ROC) curve evaluates a classi fi er ’ s ability to distinguish between two classes by plotting the true positive rate (recall) against the false positive rate over varying thresholds.”
EDLA-EFDS: A Novel Ensemble Deep Learning Approach For Electrical Fault Detection Systems
Elmasry, W., & Wadi, M. (2022) · Electric Power Systems Research · doi:10.1016/j.epsr.2022.107834 · Cited by 42
A Novel Deep Learning Approach for Anomaly Detection of Time Series Data
Ji, Z., Gong, J., & Feng, J. (2021) · Scientific Programming · doi:10.1155/2021/6636270 · Cited by 59
“As reported in previous studies, MCC produces a more informative and truthful score in evaluating binary classifications, particularly for the imbalanced data [60, 61].”
ResultSahu, Y., Tripathi, A., Gupta, R. K., Gautam, P., Pateriya, R. K., & Gupta, A. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13807-x · Cited by 54
“The metrics used are: Accuracy [20], Sensitivity [14], Specificity [14] and Precision [22].”
MethodHindi fake news detection using transformer ensembles
Praseed, A., Rodrigues, J., & Thilagam, P. S. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105731 · Cited by 46
Robust IoT Malware Detection and Classification Using Opcode Category Features on Machine Learning
Lee, H., Kim, S., Baek, D.-H., Kim, D., & Hwang, D. (2023) · IEEE Access · doi:10.1109/access.2023.3247344 · Cited by 54
“As performance metrics, accuracy (ACC), true positive rate (TPR), false positive rate (FPR), AUC-ROC, AUC-PR, F1-score (F1), and Matthews Correlation Coefficient (MCC) [50], precision (PRE), recall (REC) are chosen [47].”
BackgroundRecurrent Trend Predictive Neural Network for Multi-Sensor Fire Detection
Nakıp, M., Guzelis, C., & Yildiz, O. (2021) · IEEE Access · doi:10.1109/access.2021.3087736 · Cited by 42
Automated Prediction of Cardiorespiratory Deterioration in Patients With Single Ventricle
Rusin, C. G., Acosta, S. I., Vu, E. L., Ahmed, M., Brady, K. M., & Penny, D. J. (2021) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2021.04.072 · Cited by 42
Herrera‐Bravo, J., Belén, L. H., Farías, J. G., & Beltrán, J. F. (2021) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2021.107452 · Cited by 41
Comparative study on landslide susceptibility mapping based on unbalanced sample ratio
Li, T., Yu, X., Jiang, W., & Zhou, J. (2023) · Scientific Reports · doi:10.1038/s41598-023-33186-z · Cited by 45
Predicting soil moisture conditions across a heterogeneous boreal catchment using terrain indices
Larson, J., Lidberg, W., Ågren, A. M., & Laudon, H. (2022) · Hydrology and earth system sciences · doi:10.5194/hess-26-4837-2022 · Cited by 53
“9): For unbalanced datasets such as this, MCC is the best measure of model performance (Boughorbel et al., 2017).”
PTML Model of Enzyme Subclasses for Mining the Proteome of Biofuel Producing Microorganisms
Concu, R., Cordeiro, M. N. D. S., Munteanu, C. R., & González‐Díaz, H. (2019) · Journal of Proteome Research · doi:10.1021/acs.jproteome.8b00949 · Cited by 40
Raza, M. A., Hussain, F. K., Hussain, O. K., Zhao, M. H., & Rehman, Z. U. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.06.022 · Cited by 43
Brown, P., Tan, A.-C., El-Esawi, M. A., Liehr, T., Blanck, O., Gladue, D. P., Almeida, G. M. D. F., Cernava, T., Sorzano, C. Ó. S., Yeung, A. W. K., et al. (2019) · Database · doi:10.1093/database/baz085 · Cited by 58
“(id, title and abstract) (b) annotation data and (c) sample result file for evaluation function; (5) result file upload—allows input of result files for”
Classification of hyperspectral endocrine tissue images using support vector machines
Maktabi, M., Köhler, H., Ivanova, M., Neumuth, T., Rayes, N., Seidemann, L., Sucher, R., Jansen‐Winkeln, B., Gockel, I., Barberio, M., et al. (2020) · International Journal of Medical Robotics and Computer Assisted Surgery · doi:10.1002/rcs.2121 · Cited by 40
AI Algorithm for Mode Classification of PCF-SPR Sensor Design
Goswami, M., Khare, P., & Shakya, S. (2023) · Plasmonics · doi:10.1007/s11468-023-01997-5 · Cited by 48
Krauss, E., Haberer, J., Maurer, O., Barreto, G., Drakopanagiotakis, F., Degen, M., Seeger, W., & Günther, A. (2019) · Journal of Clinical Medicine · doi:10.3390/jcm8101698 · Cited by 42
“In essence, the MCC is a correlation between the observed and predicted binary classification, where a value of +1 represents a perfect condition, 0 represents no better than a random prediction, and −1 indicates total disagreement between the prediction and observation [20].”
Background
Metadata
- Authors
- Boughorbel, S., Jarray, F., & El-Anbari, M.
- Affiliations
- Sidra Medical and Research Center (QA)
- Conservatoire National des Arts et Métiers (FR)
- Corresponding author
- Boughorbel, Sabri
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2017-06-02
- Volume
- 12
- Issue
- 6
- Pages
- e0177678
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5456046
- OpenAlex
- W2620760558
- Cited by
- 1,222
- References
- 29
- Citations by year
- 2026: 90 · 2025: 104 · 2024: 111 · 2023: 163 · 2022: 179 · 2021: 224 · 2020: 165 · 2019: 120 · 2018: 64 · 2017: 2
- FWCI
- 57.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Qatar Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Keywords
- class imbalance, Matthews correlation coefficient, Fréchet derivative, binary classification, biomedical classification
- MeSH terms
- Algorithms, Bayes Theorem, Models, Theoretical, Computational Biology, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.