Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Data imbalance is frequently encountered in biomedical applications. Resampling techniques can be used in binary classification to tackle this issue. However such solutions are not desired when the number of samples in the small class is limited. Moreover the use of inadequate performance metrics, such as accuracy, lead to poor generalization results because the classifiers tend to predict the largest size class. One of the good approaches to deal with this issue is to optimize performance metrics that are designed to handle data imbalance. Matthews Correlation Coefficient (MCC) is widely used in Bioinformatics as a performance metric. We are interested in developing a new classifier based on the MCC metric to handle imbalanced data. We derive an optimal Bayes classifier for the MCC metric using an approach based on Frechet derivative. We show that the proposed algorithm has the nice theoretical property of consistency. Using simulated data, we verify the correctness of our optimality result by searching in the space of all possible binary classifiers. The proposed classifier is evaluated on 64 datasets from a wide range data imbalance. We compare both classification performance and CPU efficiency for three classifiers: 1) the proposed algorithm (MCC-classifier), the Bayes classifier with a default threshold (MCC-base) and imbalanced SVM (SVM-imba). The experimental evaluation shows that MCC-classifier has a close performance to SVM-imba while being simpler and more efficient.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“since it has been shown to present a balanced score when dealing with imbalanced data [11, 6] as is our case with fewer manipulated images.”
From BioFors: A Large Biomedical Image Forensics Dataset (2021) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399
A Probabilistic Theory of Pattern Recognition
Devroye, L., Györfi, L., & Lugosi, G. (1996) · Stochastic Modelling and Applied Probability · doi:10.1007/978-1-4612-0711-5 · Cited by 3,274
Assessing the accuracy of prediction algorithms for classification: an overview
Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A. F., & Nielsen, H. (2000) · Bioinformatics · doi:10.1093/bioinformatics/16.5.412 · Cited by 2,111
A sequential algorithm for training text classifiers
Lewis, D., & Gale, W. A. (1994) · arXiv (Cornell University) · doi:10.5555/188490.188495 · Cited by 1,731
A Sequential Algorithm for Training Text Classifiers
Lewis, D. D., & Gale, W. A. (1994) · SIGIR ’94 · doi:10.1007/978-1-4471-2099-5_1 · Cited by 1,208
Applying Support Vector Machines to Imbalanced Datasets
Akbani, R., Kwek, S., & Japkowicz, N. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30115-8_7 · Cited by 1,134
Yang, J., Roy, A., & Zhang, Y. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt447 · Cited by 1,035
MAQC Consortium (2010) · Nature Biotechnology · doi:10.1038/nbt.1665 · Cited by 889
Imbalanced Learning: Foundations, Algorithms, and Applications
(2013) · Wiley eBooks · doi:10.1002/9781118646106 · Cited by 831
Cited by
Items 401–500 of 1,000
Lovrić, M., Malev, O., Klobučar, G. I. V., Kern, R., Liu, J. J., & Lučić, B. (2021) · Molecules · doi:10.3390/molecules26061617 · Cited by 20
“, toxic compounds) during model training we employed the Matthews correlation coefficient (MCC) [28,34] as a scoring function during the model optimization.”
MethodSpindle-AI: Sleep Spindle Number and Duration Estimation in Infant EEG
Wei, L., Ventura, S., Mathieson, S., Boylan, G. B., Lowery, M. M., & Mooney, C. (2021) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3097815 · Cited by 24
Čeponis, D., & Goranin, N. (2020) · Applied Sciences · doi:10.3390/app10072373 · Cited by 24
Galdrán, A., Chelbi, J., Kobbi, R., Dolz, J., Lombaert, H., Ayed, I. B., & Chakor, H. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.2.34 · Cited by 22
Ahmed, M. U., & Begum, S. (2020) · Frontiers in Sustainable Cities · doi:10.3389/frsc.2020.00034 · Cited by 18
“77 Negative (TN), False Positive (FP) and False Negative (FN) and the formula can be express as follows in eq2 (Boughorbel et al., 2017), the MCC results are presented in Table 7.”
BackgroundData-Based Windstorm Type Identification Algorithm and Extreme Wind Speed Prediction
Cui, W., Ma, T., Zhao, L., & Ge, Y. (2021) · Journal of Structural Engineering · doi:10.1061/(asce)st.1943-541x.0002954 · Cited by 21
“However, only the F1-score of SVM and LR on three types of storms is higher than 0.6.”
ResultInfluentialMelanoma Recognition by Fusing Convolutional Blocks and Dynamic Routing between Capsules
Pérez, E., & Ventura, S. (2021) · Cancers · doi:10.3390/cancers13194974 · Cited by 21
“Regarding the evaluation metrics, Matthews Correlation Coefficient (MCC) and the area under the curve (AUC) values for receiver operator characteristic (ROC) were used to measure the predictive performance of the models, which are commonly applied in Bioinformatics [54,55].”
MethodComparison of algorithms for error prediction in manufacturing with automl and a cost-based metric
Gerling, A., Ziekow, H., Hess, A., Schreier, U., Seiffer, C., & Abdeslam, D. (2022) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-021-01890-0 · Cited by 11
“The MCC is used for different class sizes which is necessary in our case (Boughorbel et al., 2017).”
MethodKotowski, K., Kucharski, D., Machura, B., Adamski, S., Becker, B. G., Krason, A., Zarudzki, Ł., Tessier, J., & Nalepa, J. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106378 · Cited by 21
Zhao, P. F., Guo, S., Li, Y., Yang, S., & Ren, X. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.107595 · Cited by 26
In silico model for mutagenicity (Ames test), taking into account metabolism
Vian, M., Raitano, G., Roncaglioni, A., & Benfenati, E. (2018) · Mutagenesis · doi:10.1093/mutage/gey045 · Cited by 18
Segmentation-based Deep Learning Fundus Image Analysis
Wu, Q., & Cheddad, A. (2019) · International Conference on Image Processing Theory, Tools and Applications (IPTA) · doi:10.1109/ipta.2019.8936078 · Cited by 20
“The MCC is a balanced measure which can be used even if the classes are of very different sizes [16].”
BackgroundCuBlock: a cross-platform normalization method for gene-expression microarrays
Junet, V., Farrés, J., Mas, J. M., & Daura, X. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab105 · Cited by 23
“, 2003), which was not completely unexpected—for example, Hubbell et al. (2002) found that simple averaging performed comparably to more robust approaches from the Affymetrix Micro Array Suite under low-noise conditions.”
MethodConsidering Non-sequential Control Flows for Process Prediction with Recurrent Neural Networks
Metzger, A., & Neubauer, A. (2018) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa.2018.00051 · Cited by 18
“, if the positive and negative classes are of different size [20].”
BackgroundBehr, M., Noseworthy, M. D., & Kumbhare, D. A. (2019) · Journal of Ultrasound in Medicine · doi:10.1002/jum.14909 · Cited by 20
“The MCC takes true-positive (TP), false-positive (FP), true-negative (TN), and false-negative (FN) results into account and is widely regarded as a balanced performance metric for binary classification.(56) Additionally, it is perfect for situations involving unbalanced classes.”
BackgroundWhata, A., Dibeco, K., Madzima, K., & Obagbuwa, I. C. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1410841 · Cited by 17
Towards robust validation strategies for EO flood maps
Landwehr, T., Dasgupta, A., & Waske, B. (2024) · Remote Sensing of Environment · doi:10.1016/j.rse.2024.114439 · Cited by 27
Ensemble models improve near-term forecasts of harmful algal bloom and biotoxin risk
Szewczyk, T. M., Aleynik, D. L., & Davidson, K. (2024) · Harmful Algae · doi:10.1016/j.hal.2024.102781 · Cited by 20
“The Matthews Correlation Coefficient (MCC) is equivalent to a Pearson correlation coefficient for two binary variables (Boughorbel et al., 2017; Cram ´ er, Harald, 1946; Matthews, 1975).”
Neural Networks and Imbalanced Learning for Data-Driven Scientific Computing With Uncertainties
Pourkamali‐Anaraki, F., & Hariri‐Ardebili, M. A. (2021) · IEEE Access · doi:10.1109/access.2021.3052680 · Cited by 21
“However, in the case of imbalanced data, this metric is misleading since the minority class holds a small effect on this measure [79].”
BackgroundSolarDiagnostics: Automatic damage detection on rooftop solar photovoltaic arrays
Li, Q., Yu, K., & Chen, D. (2021) · Sustainable Computing Informatics and Systems · doi:10.1016/j.suscom.2021.100595 · Cited by 17
Systems biology and machine learning approaches identify drug targets in diabetic nephropathy
Abedi, M., Marateb, H. R., Mohebian, M. R., Aghaee‐Bakhtiari, S. H., Nassiri, S. M., & Gheisari, Y. (2021) · Scientific Reports · doi:10.1038/s41598-021-02282-3 · Cited by 20
“Since the data is highly imbalanced, the Mat-thews correlation coefficient was used as the fitness function 71 instead of the traditional RMSE.”
MethodBartels, J., Tokgöz, K. K., Sihan, A., Fukawa, M., Otsubo, S., Li, C., Rachi, I., Takeda, K.-I., Minati, L., & Ito, H. (2022) · IEEE Access · doi:10.1109/access.2022.3156278 · Cited by 19
“Next to the MAE and accuracy, we employ the Matthews Correlation Coefficient (MCC) that ranges from -1 to 1, as it is a standard metric used on imbalanced datasets [33], [76].”
MethodSawadogo, Z., Mendy, G., Dembélé, J. M., & Ouya, S. (2022) · International Conference on Advanced Communication Technology · doi:10.23919/icact53585.2022.9728833 · Cited by 21
Towards Automated Imbalanced Learning with Deep Hierarchical Reinforcement Learning
Zha, D., Lai, K.-H., Tan, Q., Ding, S., Zou, N., & Hu, X. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557474 · Cited by 21
“Following the previous work of imbalanced classification [1, 12, 31], we use Macro-F1 and Matthews Correlation Coefficient (MCC) to evaluate the performance.”
MethodThe Pursuit of Peer Support for Opioid Use Recovery on Reddit
Balsamo, D., Bajardi, P., Morales, G. D. F., Monti, C., & Schifanella, R. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22122 · Cited by 31
“Based on the best validation performance of AUC = 0.942 and Matthew’s correlation coefficient (Boughorbel, Jarray, and El-Anbari 2017) MCC = 0.847 we select the pipeline consisting of time-expression masking, tf-idf weighting, and logistic regression model as the most suitable for our task.”
MethodInfluentialLeón, L., Ortíz, A., Ezquerro, S., & Tejerina, D. (2023) · Meat Science · doi:10.1016/j.meatsci.2023.109348 · Cited by 17
Cruz-Guerrero, I. A., Campos-Delgado, D. U., Mejia-Rodriguez, A. R., León, R., Ortega, S., Fabelo, H. A., Camacho, R., Plaza, M. D. L. L., & Callicó, G. M. (2024) · Healthcare Technology Letters · doi:10.1049/htl2.12084 · Cited by 21
Gnekpe, C., Tchuente, D., Nyawa, S., & Dey, P. K. (2024) · Journal of Business Research · doi:10.1016/j.jbusres.2024.114821 · Cited by 20
Efficient plastic detection in coastal areas with selected spectral bands
Pérez-García, Á., van Emmerik, T. H. M., Mata, A., Tasseron, P., & Lopez, J. F. (2024) · Marine Pollution Bulletin · doi:10.1016/j.marpolbul.2024.116914 · Cited by 20
“The F1 score is particularly useful with uneven class distribution, giving more importance to the accuracy of the smaller classes Boughorbel et al. (2017).”
Machine and deep learning for the prediction of nutrient deficiency in wheat leaf images
Sirsat, M. S., Isla-Cernadas, D., Cernadas, E., & Fernández-Delgado, M. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113400 · Cited by 20
SVM-SulfoSite: A support vector machine based predictor for sulfenylation sites
AL-barakati, H. J., McConnell, E. W., Hicks, L. M., Poole, L. B., Newman, R. H., & KC, D. B. (2018) · Scientific Reports · doi:10.1038/s41598-018-29126-x · Cited by 18
Pum, J. K. W. (2019) · Clinica Chimica Acta · doi:10.1016/j.cca.2019.07.018 · Cited by 22
What is Test Accuracy? Comparing Unitary Accuracy Metrics for Cognitive Screening Instruments
Larner, A. J. (2019) · Neurodegenerative Disease Management · doi:10.2217/nmt-2019-0017 · Cited by 17
Jimoh, I. A., Ismaila, I., & Olalere, M. (2019) · International Conference on Electronics, Computer and Computation · doi:10.1109/icecco48375.2019.9043233 · Cited by 20
“Bioinformatics as a performance metric [13] .”
BackgroundAmador, T., Saturnino, S. F., Veloso, A., & Ziviani, N. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102283 · Cited by 20
Resampling-based cost loss attention network for explainable imbalanced diabetic retinopathy grading
Li, H., Dong, X., Shen, W., Ge, F., & Li, H. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105970 · Cited by 20
Francini, M., Salvo, C., Viscomi, A., & Vitale, A. (2022) · Remote Sensing · doi:10.3390/rs14174279 · Cited by 19
Burd, P. R., Karl Mannheim, M., März, T., Ringholz, J., Kappes, A., & Kadler, M. (2018) · Astronomische Nachrichten · doi:10.1002/asna.201813505 · Cited by 19
“The Matthews correlation coefficient (MCC), see Eq.11, evaluates the network’s performance when dealing with sample sizes which widely differ in range, see (Matthews, 1975) and (Boughorbel et al., 2017).”
BackgroundGalanis, K. A., Nastou, K. C., Papandreou, N. C., Petichakis, G. N., Pigis, D. G., & Iconomidou, V. A. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/833418 · Cited by 21
“Out of these metrics, significant attention was given to MCC, since it is generally regarded as the best performance metric for binary classifiers [68, 69].”
BackgroundRanjbarzadeh, R., Zarbakhsh, P., Caputo, A., Tırkolaee, E. B., & Bendechache, M. (2022) · Research Square · doi:10.21203/rs.3.rs-2203596/v1 · Cited by 17
A framework to build accurate Convolutional Neural Network models for melanoma diagnosis
Pérez, E., & Ventura, S. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110157 · Cited by 20
Modelling Hail Hazard Over Italy with Era5 Large-Scale Variables
Torralba, V., Hénin, R., Cantelli, A., Scoccimarro, E., Materia, S., Manzato, A., & Gualdi, S. (2022) · Weather and Climate Extremes · doi:10.1016/j.wace.2022.100535 · Cited by 20
iIL13Pred: improved prediction of IL-13 inducing peptides using popular machine learning classifiers
Arora, P., Periwal, N., Goyal, Y., Sood, V., & Kaur, B. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05248-6 · Cited by 15
“MCC is known to be the preferred metric in case of an unbalanced data [21].”
BackgroundDonahue, K., Kimball, J. S., Du, J., Bunt, F., Colliander, A., Moghaddam, M., Johnson, J. V., Kim, Y., & Rawlins, M. A. (2023) · Frontiers in Big Data · doi:10.3389/fdata.2023.1243559 · Cited by 20
Diverse misinformation: impacts of human biases on detection of deepfakes on networks
Lovato, J., St-Onge, J., Harp, R., Salazar Lopez, G., Rogers, S. P., Haq, I. U., Hébert-Dufresne, L., & Onaolapo, J. (2024) · npj Complexity · doi:10.1038/s44260-024-00006-y · Cited by 16
Diagnosis of multiple sclerosis using multifocal ERG data feature fusion
López-Dorado, A., Perez, J. L., Rodrigo, M. J., Miguel, J. M., Ortiz, M., de Santiago, L., López-Guillén, E., Blanco, R., Cavalliere, C., Sánchez‐Morla, E. M., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.006 · Cited by 17
A Retrospective Analysis ofb-Value Changes Preceding Strong Earthquakes
DeSalvio, N. D., & Rudolph, M. L. (2021) · Seismological Research Letters · doi:10.1785/0220210149 · Cited by 16
BayesBeat: Reliable Atrial Fibrillation Detection from Noisy Photoplethysmography Data
Das, S. S. S., Shanto, S. K., Rahman, M., Islam, M. S., Rahman, A. H., Masud, M. M., & Ali, M. E. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3517247 · Cited by 30
“Though AUC is commonly used in the prior works, it can give misleading performance representation for class imbalanced dataset; whereas MCC is particularly effective in assessing the performance of a binary classifier in case of imbalanced data [5].”
MethodPuranam, P., Shrestha, Y. R., He, V. F., & von Krogh, G. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3140617 · Cited by 23
Automating parameter selection to avoid implausible biological pathway models
Magnano, C. S., & Gitter, A. (2021) · npj Systems Biology and Applications · doi:10.1038/s41540-020-00167-1 · Cited by 15
Wani, M. A., & Roy, K. K. (2021) · Molecular Diversity · doi:10.1007/s11030-021-10238-y · Cited by 17
Yan, C., Li, M., Ma, J., Liao, Y., Luo, H., Wang, J., & Luo, J. (2021) · Current Bioinformatics · doi:10.2174/1574893616666210624130124 · Cited by 15
Al‐Bander, B., Yas, Q. M., Mahdi, H. F., & Al‐Hamd, R. K. S. (2021) · TURKISH JOURNAL OF ELECTRICAL ENGINEERING & COMPUTER SCIENCES · doi:10.3906/elk-2103-65 · Cited by 17
Assessment of spatiotemporal dynamics of diurnal fog occurrence in subtropical montane cloud forests
Li, H., Lo, M., Juang, J., Wang, J., & Huang, C. (2022) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2022.108899 · Cited by 17
López, C. D., González, D. M., Vidaki, A., & Kayser, M. (2022) · Frontiers in Microbiology · doi:10.3389/fmicb.2022.886201 · Cited by 17
“TheMCC performancemetric allowed us for fair comparisons of the validated ML methods for both non-augmented and augmented data, since MCC is independent of data imbalance (Boughorbel et al., 2017; Ballabio et al., 2018).”
MethodQuality Prediction of Continuous Casting Slabs Based on Weighted Extreme Learning Machine
Liu, Y., Lu, H., Zhang, H., Wu, X., Zhong, Y., & Lei, Z. (2022) · IEEE Access · doi:10.1109/access.2022.3192541 · Cited by 15
Machine Learning to Assess Relatedness: The Advantage of Using Firm‐Level Data
Albora, G., & Zaccaria, A. (2022) · Complexity · doi:10.1155/2022/2095048 · Cited by 23
Żygierewicz, J., Janik, R. A., Podolak, I. T., Drozd, A., Malinowska, U., Poziomska, M., Wojciechowski, J., Ogniewski, P., Niedbalski, P., Terczyńska, I., et al. (2022) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac8b38 · Cited by 19
Herfurth, K., Harpaz, Y., Roesch, J., Mueller, N., Walther, K., Kaltenhäuser, M., Pauli, E., Goldstein, A., Hamer, H. M., Buchfelder, M., et al. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.996989 · Cited by 17
“As Kappa can be inaccurate in case of imbalanced distribution of classes (Delgado and Tibau, 2019), we additionally calculated Matthews Correlation Coefficient as a performance metric (Boughorbel et al., 2017).”
MethodTissue Classification of Breast Cancer by Hyperspectral Unmixing
Jong, L.-J. S., Post, A. L., Veluponnar, D., Geldof, F., Sterenborg, H. J. C. M., Ruers, T. J. M., & Dashtbozorg, B. (2023) · Cancers · doi:10.3390/cancers15102679 · Cited by 20
“The MCC can be considered as a more robust alternative for accuracy in the case of an imbalanced dataset [34].”
BackgroundHalder, A. K., Moura, A. S., & Cordeiro, M. N. D. S. (2023) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2023.164337 · Cited by 16
Nonparametric statistics. Part 3. Correlation coefficients
Nikitina, M. A., & Chernukha, I. M. (2023) · Theory and practice of meat processing · doi:10.21323/2414-438x-2023-8-3-237-251 · Cited by 20
Sakib, N., Paul, T., Ahmed, M. T., Momin, K. A., & Barua, S. (2023) · Multimodal Transportation · doi:10.1016/j.multra.2023.100108 · Cited by 19
Casal-Guisande, M., Comesaña-Campos, A., Fernández, M. N., Torres‐Durán, M., & Fernández‐Villar, A. (2024) · Biomedicines · doi:10.3390/biomedicines12040854 · Cited by 15
Multiclass deep learning model for predicting lung diseases based on honey badger algorithm
Askr, H., Moawad, M., Darwish, A., & Hassanien, A. E. (2024) · International Journal of Information Technology · doi:10.1007/s41870-024-02046-y · Cited by 21
“The proposed model’s Geometric Mean (GM) achieved 0.996, indicating high efficiency, as it is an effective statistic for assessing classifier performance [ 18].”
InfluentialA Univariate Feature Selection Approach for Finding Key Factors of Restaurant Business
Subho, M. R. H., Chowdhury, M. R., Chaki, D., Islam, S., & Rahman, M. M. (2019) · IEEE Region 10 Symposium (TENSYMP) · doi:10.1109/tensymp46218.2019.8971127 · Cited by 15
2-D chemical structure image-based in silico model to predict agonist activity for androgen receptor
Yu, M., Lee, J., Lee, Y., & Na, D. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-03588-1 · Cited by 16
Novel prediction models for hyperketonemia using bovine milk Fourier-transform infrared spectroscopy
Walleser, E., Reyes, J. F. M., Anklam, K., Pralle, R. S., White, H. M., Unger, S., Panne, N. C., Kammer, M., Plattner, S., & Döpfer, D. (2023) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2023.105860 · Cited by 16
Alrumaih, T. N. I., & Alenazi, M. J. F. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3390691 · Cited by 19
Predicting COPD Readmission: An Intelligent Clinical Decision Support System
López-Canay, J., Casal-Guisande, M., Pinheira, A. G., Golpe, R., Comesaña-Campos, A., Fernández-García, A., Represas‐Represas, C., & Fernández‐Villar, A. (2025) · Diagnostics · doi:10.3390/diagnostics15030318 · Cited by 16
A Novel Short Text Clustering Model Based on Grey System Theory
Fidan, H., & Yüksel, M. E. (2019) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-019-04191-0 · Cited by 15
“In some studies, it is emphasized that MCC method is more suitable for performance evaluation of unbalanced classifications [72, 73].”
MethodLissabet, J. F. B., Belén, L. H., & Farías, J. G. (2020) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-019-00356-5 · Cited by 12
DeEPn: a deep neural network based tool for enzyme functional annotation
Semwal, R., Aier, I., Tyagi, P., & Varadwaj, P. K. (2020) · Journal of Biomolecular Structure and Dynamics · doi:10.1080/07391102.2020.1754292 · Cited by 13
“To evaluate the model performance, F1-score is a more robust metric compared to accuracy; however, in the case of unbalanced test cases, the F1-score sometimes overestimate the classification performance (Boughorbel et al., 2017).”
MethodRoad Surface Wetness Quantification Using a Capacitive Sensor System
Döring, J., Beering, A., Scholtyssek, J., & Krieger, K.-L. (2021) · IEEE Access · doi:10.1109/access.2021.3121099 · Cited by 16
“It represents a metric that has been proven to be appropriate for performance evaluation and classifier optimization in case of imbalanced data sets [53], [60].”
BackgroundMahboobeh, D. J., Dias, S. B., Khandoker, A. H., & Hadjileontiadis, L. J. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.857249 · Cited by 14
“, not being normalized and not being symmetric (when swapping positive and negative cases), and is considered as a balanced measure, even when the classes are of very different sizes (Boughorbel et al., 2017).”
BackgroundBriand, J., Deguire, S., Gaudet, S., & Bieuzen, F. (2022) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2022.896828 · Cited by 25
“An MCC of 0 would be considered a model where the predictions were made randomly (Boughorbel et al., 2017).”
BackgroundInfluentialA system-theoretic approach for image-based infectious plant disease severity estimation
Palma, D., Blanchini, F., & Montessoro, P. L. (2022) · PLoS ONE · doi:10.1371/journal.pone.0272002 · Cited by 14
Jia, J., Sun, M., Wu, G., & Qiu, W. (2022) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2023132 · Cited by 14
Feature Subset Selection using Adaptive Differential Evolution: An Application to Banking
Krishna, G. J., & Ravi, V. (2019) · Joint International Conference on Data Science & Management of Data (CODS-COMAD) · doi:10.1145/3297001.3297021 · Cited by 18
“MCC is utilized in the single objective framework for ADE.”
BackgroundMethodInfluentialIdentifying Antifreeze Proteins Based on Key Evolutionary Information
Sun, S., Ding, H., Wang, D., & Han, S. (2020) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2020.00244 · Cited by 14
Classification and Regression Tree with Resampling for Classifying Imbalanced Data
(2018) · International Journal of Machine Learning and Computing · doi:10.18178/ijmlc.2018.8.4.708 · Cited by 14
Comprehensive Immune Monitoring of Clinical Trials to Advance Human ImmunotherapyPreprint
Hartmann, F. J., Babdor, J., Gherardini, P. F., Amir, E.-A. D., Jones, K. B., Sahaf, B., Marquez, D. M., Krutzik, P. O., O'Donnell, E. A., Sigal, N., et al. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/489765 · Cited by 14
Zazzaro, G., & Pavone, L. (2022) · Biomedicines · doi:10.3390/biomedicines10071491 · Cited by 13
Detection of COVID-19 Infection in CT and X-ray images using transfer learning approach
Tiwari, A., Tripathi, S., Pandey, D. C., Sharma, N., & Sharma, S. (2022) · Technology and Health Care · doi:10.3233/thc-220114 · Cited by 14
A Neural Network-based Cloud Mask for PREFIRE and Evaluation with Simulated Observations
Bertossa, C., L’Ecuyer, T., Merrelli, A. J., Huang, X., & Chen, X. (2023) · Journal of Atmospheric and Oceanic Technology · doi:10.1175/jtech-d-22-0023.1 · Cited by 19
“is applicable even if classes are of very different size (Boughorbel et al., 2017).”
BackgroundEffect of Missing Data Types and Imputation Methods on Supervised Classifiers: An Evaluation Study
Gabr, M. I., Helmy, Y., & Elzanfaly, D. S. (2023) · Big Data and Cognitive Computing · doi:10.3390/bdcc7010055 · Cited by 23
Mapping user behaviors to identify professional accounts in Ethereum using semi-supervised learning
Valadares, J. A., Villela, S. M., Bernardino, H. S., Gonçalves, G. D., & Vieira, A. B. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120438 · Cited by 12
Deep face segmentation for improved heart and respiratory rate estimation from videos
Fiedler, M.-A., Werner, P., Rapczyński, M., & Al-Hamadi, A. K. (2023) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-023-04607-8 · Cited by 15
“It has been shown in other works that it is a more reliable metric for evaluating classification performance (Chicco and Jurman 2020; Chicco et al. 2021) and particularly suitable for handling data imbalance (Boughorbel et al. 2017; Zhu 2020).”
BackgroundMetzger, A., Kley, T., Rothweiler, A., & Pohl, K. (2023) · Information Systems · doi:10.1016/j.is.2023.102254 · Cited by 21
“As contingency table metric, we use the Matthews Correlation Coefficient (MCC), which is robust against class imbalances [77].”
MethodBriciu, A., Czibula, G., & Lupea, M. (2023) · Procedia Computer Science · doi:10.1016/j.procs.2023.10.149 · Cited by 23
Casal-Guisande, M., Fernández‐Villar, A., Mosteiro, M., Comesaña-Campos, A., Cerqueiro-Pequeño, J., & Torres‐Durán, M. (2024) · Digital Health · doi:10.1177/20552076241272632 · Cited by 16
Talukdar, N., Kakati, A., Barman, U., Medhi, J. P., Sarma, K. K., Barman, G., & Choudhury, B. K. (2025) · Innovative Practice in Breast Health · doi:10.1016/j.ibreh.2025.100038 · Cited by 15
Krishnan, G. S., & Kamath, S. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-91947-8_13 · Cited by 15
Relevant features for gender classification in NIR periocular images
Viedma, I. A., Tapia, J. E., Iturriaga, A., & Busch, C. (2019) · IET Biometrics · doi:10.1049/iet-bmt.2018.5233 · Cited by 14
A Chi-MIC Based Adaptive Multi-Branch Decision Tree
Ye, J., Yang, J., Yu, J., Tan, S., Luo, F., Yuan, Z., & Chen, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3077125 · Cited by 15
“Matthews Correlation Coefficient ( MCC ) [41] and G-mean [42] are independent of the class distribution in the data set, but the MCC only for binary clas-sification, and the G-mean tend to be zero while all samples of one minority class are misclassified.”
BackgroundInfluentialZhou, Y., Maître, R., Hupel, M., Trotoux, G., Penguilly, D., Mariette, F., Bousset, L., Chèvre, A., & Parisey, N. (2021) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2021.106303 · Cited by 16
MR-UNet Commodity Semantic Segmentation Based on Transfer Learning
Wu, Z., Zhao, L., & Zhang, H.-X. (2021) · IEEE Access · doi:10.1109/access.2021.3130578 · Cited by 15
“The coefficient takes into account , , and , and is generally considered as a measure of equilibrium.”
BackgroundFaizabadi, A. R., Zaki, H. F. M., Abidin, Z. Z., Hashim, N. N. W. N., & Husman, M. A. (2022) · IEEE Access · doi:10.1109/access.2022.3192520 · Cited by 15
Machine Learning Based Resilience Testing of an Address Randomization Cyber Defense
Mani, G., Haliem, M., Bhargava, B., Manickam, I., Kochpatcharin, K., Kim, M., Vugrin, E. D., Wang, W., Jenkins, C., Angın, P., et al. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3234561 · Cited by 17
Rainey, C., Villikudathil, A. T., McConnell, J., Hughes, C., Bond, R., & McFadden, S. L. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000229 · Cited by 14
“MCC = ((TPxTN)(FPxFN))/ p ((TP+FP)x(TP+FN)x(TN+FP)x(TN+FN)) vi. Area Under Curve (AUC): The area under the Receiver Operating Characteristic (ROC) curve helps to compare the performance of different ML classifier models [43].”
Method
Metadata
- Authors
- Boughorbel, S., Jarray, F., & El-Anbari, M.
- Affiliations
- Sidra Medical and Research Center (QA)
- Conservatoire National des Arts et Métiers (FR)
- Corresponding author
- Boughorbel, Sabri
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2017-06-02
- Volume
- 12
- Issue
- 6
- Pages
- e0177678
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5456046
- OpenAlex
- W2620760558
- Cited by
- 1,222
- References
- 29
- Citations by year
- 2026: 90 · 2025: 104 · 2024: 111 · 2023: 163 · 2022: 179 · 2021: 224 · 2020: 165 · 2019: 120 · 2018: 64 · 2017: 2
- FWCI
- 57.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Qatar Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Keywords
- class imbalance, Matthews correlation coefficient, Fréchet derivative, binary classification, biomedical classification
- MeSH terms
- Algorithms, Bayes Theorem, Models, Theoretical, Computational Biology, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.