Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Data imbalance is frequently encountered in biomedical applications. Resampling techniques can be used in binary classification to tackle this issue. However such solutions are not desired when the number of samples in the small class is limited. Moreover the use of inadequate performance metrics, such as accuracy, lead to poor generalization results because the classifiers tend to predict the largest size class. One of the good approaches to deal with this issue is to optimize performance metrics that are designed to handle data imbalance. Matthews Correlation Coefficient (MCC) is widely used in Bioinformatics as a performance metric. We are interested in developing a new classifier based on the MCC metric to handle imbalanced data. We derive an optimal Bayes classifier for the MCC metric using an approach based on Frechet derivative. We show that the proposed algorithm has the nice theoretical property of consistency. Using simulated data, we verify the correctness of our optimality result by searching in the space of all possible binary classifiers. The proposed classifier is evaluated on 64 datasets from a wide range data imbalance. We compare both classification performance and CPU efficiency for three classifiers: 1) the proposed algorithm (MCC-classifier), the Bayes classifier with a default threshold (MCC-base) and imbalanced SVM (SVM-imba). The experimental evaluation shows that MCC-classifier has a close performance to SVM-imba while being simpler and more efficient.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“since it has been shown to present a balanced score when dealing with imbalanced data [11, 6] as is our case with fewer manipulated images.”
From BioFors: A Large Biomedical Image Forensics Dataset (2021) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399
A Probabilistic Theory of Pattern Recognition
Devroye, L., Györfi, L., & Lugosi, G. (1996) · Stochastic Modelling and Applied Probability · doi:10.1007/978-1-4612-0711-5 · Cited by 3,274
Assessing the accuracy of prediction algorithms for classification: an overview
Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A. F., & Nielsen, H. (2000) · Bioinformatics · doi:10.1093/bioinformatics/16.5.412 · Cited by 2,111
A sequential algorithm for training text classifiers
Lewis, D., & Gale, W. A. (1994) · arXiv (Cornell University) · doi:10.5555/188490.188495 · Cited by 1,731
A Sequential Algorithm for Training Text Classifiers
Lewis, D. D., & Gale, W. A. (1994) · SIGIR ’94 · doi:10.1007/978-1-4471-2099-5_1 · Cited by 1,208
Applying Support Vector Machines to Imbalanced Datasets
Akbani, R., Kwek, S., & Japkowicz, N. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30115-8_7 · Cited by 1,134
Yang, J., Roy, A., & Zhang, Y. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt447 · Cited by 1,035
MAQC Consortium (2010) · Nature Biotechnology · doi:10.1038/nbt.1665 · Cited by 889
Imbalanced Learning: Foundations, Algorithms, and Applications
(2013) · Wiley eBooks · doi:10.1002/9781118646106 · Cited by 831
Cited by
Items 301–400 of 1,000
Recognizing white blood cells with local image descriptors
López-Puigdollers, D., Traver, V. J., & Pla, F. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.08.029 · Cited by 28
A critical analysis of metrics used for measuring progress in artificial intelligencePreprint
Blagec, K., Dorffner, G., Moradi, M., & Samwald, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02577 · Cited by 79
“(Boughorbel et al. 2017; Chicco and Jurman 2020) MCC is based on the entire confusion matrix and takes on values in the interval [-1, +1], where the values of -1 and +1 imply perfect misclassification and perfect classification, respectively, and 0 implies random predictions.”
BackgroundYan, F., Zhang, M., Wei, B., Ren, K., & Jiang, W. (2024) · Alexandria Engineering Journal · doi:10.1016/j.aej.2024.08.103 · Cited by 31
Fresard, M. E., Erices, R., Bravo, M. L., Cuello, M. A., Owen, G. I., Ibáñez, C., & Rodriguez-Fernández, M. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2019.2943499 · Cited by 27
Ren, N., Ito, S., Hafizi, H., Beggs, J. M., & Stevenson, I. H. (2020) · Journal of Neurophysiology · doi:10.1152/jn.00066.2020 · Cited by 33
Guttman, Y., & Kerem, Z. (2022) · Journal of Agricultural and Food Chemistry · doi:10.1021/acs.jafc.2c00237 · Cited by 31
Sekajugo, J., Kagoro‐Rugunda, G., Mutyebere, R., Kabaseke, C., Namara, E., Dewitte, O., Kervyn, M., & Jacobs, L. (2022) · Environmental Research Letters · doi:10.1088/1748-9326/ac5bb5 · Cited by 33
Watzig, C., Schaumberger, A., Klingler, A., Dujakovic, A., Atzberger, C., & Vuolo, F. (2023) · Remote Sensing of Environment · doi:10.1016/j.rse.2023.113577 · Cited by 26
Deep learning applications on cybersecurity: A practical approach
Miranda-García, A., Rego, A. Z., Pastor-López, I., Sanz, B., Tellaeche, A., de la Puerta, J. G., & Bringas, P. G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126904 · Cited by 40
Wang, D., Tong, X., & Wang, Y. (2020) · Resources Policy · doi:10.1016/j.resourpol.2020.101593 · Cited by 29
J Regularization Improves Imbalanced Multiclass Segmentation
Peña, F. A. G., Fernández, P. D. M., Tarr, P. T., Ren, T. I., Meyerowitz, E. M., & Cunha, A. (2020) · CaltechAUTHORS (California Institute of Technology) · Cited by 26
Maruf, A. A., Fahim, S. H., Bashar, R., Rumy, R. A., Chowdhury, S. I., & Aung, Z. (2024) · IEEE Access · doi:10.1109/access.2024.3426041 · Cited by 30
“The Matthews correlation coefficient (MCC) [62] evaluates the accuracy of binary classifications for unbalanced data.”
MethodImproving Paraphrase Detection with the Adversarial Paraphrasing Task
Nighojkar, A., & Licato, J. (2021) · doi:10.18653/v1/2021.acl-long.552 · Cited by 50
“Note that since the number of MI and non-MI sentences in all the datasets is imbalanced, Matthew’s Correlation Coefficient (MCC) is a more appropriate performance measure than accuracy (Boughorbel et al., 2017).”
Casanova-Alvarez, O., Helguera, A. M., Cabrera‐Pérez, M. Á., Molina, R., & Molina, C. G. (2021) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.0c01439 · Cited by 25
A hybrid model for efficient cervical cell classification
Sabeena, K., & Gopakumar, C. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.103288 · Cited by 27
Machine Learning for Outcome Prediction in First-Line Surgery of Prolactinomas
Huber, M., Luedi, M. M., Schubert, G. A., Musahl, C., Tortora, A., Frey, J., Beck, J., Mariani, L., Christ, E., & Andereggen, L. (2022) · Frontiers in Endocrinology · doi:10.3389/fendo.2022.810219 · Cited by 25
“true and false both positives and negatives) (48); another is that it performs well on imbalanced data sets (49).”
BackgroundLöhle, M., Timpka, J., Bremer, A., Khodakarami, H., Gandor, F., Horne, M. K., Ebersbach, G., Odin, P., & Storch, A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00937-1 · Cited by 25
Zhang, Z., Xiong, P., Zhang, T., Wang, J., Zhan, J., & Zhou, Y. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz1192 · Cited by 30
“We also employed the precision-recall curve rather than the receiver operator characteristic curve because our main interest is the minor class of positive samples (51) (in this case, base pairs).”
BackgroundMethodDetection of algorithmically-generated domains: An adversarial machine learning approach
Alaeiyan, M., Parsa, S. A., Vinod, P., & Conti, M. (2020) · Computer Communications · doi:10.1016/j.comcom.2020.04.033 · Cited by 27
Implicit auditory memory in older listeners: From encoding to 6-month retention
Bianco, R., Hall, E. T. R., Pearce, M. T., & Chait, M. (2023) · Current Research in Neurobiology · doi:10.1016/j.crneur.2023.100115 · Cited by 26
“The classification was evaluated using the Matthews Correlation Coefficient (MCC) score which ranges between 1 (perfect classification) to (cid:0) 1 (total misclassification) (Boughorbel et al., 2017; Powers, 2007).”
MethodMortality Risk in Homebound Older Adults Predicted From Routinely Collected Nursing Data
Sullivan, S. S., Hewner, S., Chandola, V., & Westra, B. L. (2019) · Nursing Research · doi:10.1097/nnr.0000000000000328 · Cited by 24
“TPR is defined as the proportion of cases that are correctly predicted by themodel to be “not alive” at 12months (Equation 1; Laerd Statistics, 2015, Boughorbel et al., 2017).”
Larner, A. J. (2020) · Neurodegenerative Disease Management · doi:10.2217/nmt-2020-0003 · Cited by 25
Osama, S., Zafar, K., & Sadiq, M. U. (2020) · Diagnostics · doi:10.3390/diagnostics10110858 · Cited by 33
Pirmoradi, S., Teshnehlab, M., Zarghami, N., & Sharifi, A. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106132 · Cited by 28
Przedborski, M., Smalley, M., Thiyagarajan, S., Goldman, A. J., & Kohandel, M. (2021) · Communications Biology · doi:10.1038/s42003-021-02393-7 · Cited by 29
HCBST: An Efficient Hybrid Sampling Technique for Class Imbalance Problems
Sowah, R. A., Kuditchar, B., Mills, G. A., Acakpovi, A., Twum, R. A., Buah, G., & Agboyi, R. (2021) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3488280 · Cited by 25
“57:18 R. A. Sowah et al. MCC measures the correlation coefficient between the observations and the predictions from a classifier [6].”
BackgroundAn Unsupervised Software Fault Prediction Approach Using Threshold Derivation
Kumar, R., Chaturvedi, A., & Kailasam, L. (2022) · IEEE Transactions on Reliability · doi:10.1109/tr.2022.3151125 · Cited by 27
Predicting emergency department utilization among children with asthma using deep learning models
AlSaad, R., Malluhi, Q. M., Janahi, I. A., & Boughorbel, S. (2022) · Healthcare Analytics · doi:10.1016/j.health.2022.100050 · Cited by 32
Self-citationNSCGRN: a network structure control method for gene regulatory network inference
Liu, W., Sun, X., Yang, L., Li, K., Yu, Y., & Fu, X. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac156 · Cited by 26
Verma, D., Jansen, D., Bach, K., Poel, M., Mork, P. J., & Nijeweme-d’Hollosy, W. O. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01973-9 · Cited by 26
“The evaluation metrics in the regression task are Mean Absolute Error (MAE), R-squared score ( R 2 ) and Mean Residual (MR), while for the classification task are Mat-thews Correlation Coefficient (MCC) [48], Balanced Accuracy (BAC) [49], Sensitivity (SEN) and Specificity (SPE) [50].”
MethodInfluentialCasiano, A. S., Mehta, D., Danciu, O. C., Patel, H., Banks, L., Zaidi, A., Buckley, J., Rauscher, G. H., Schulte, L., Weller, L., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-39215-1 · Cited by 23
Assessing risk of bias in toxicological studies in the era of artificial intelligence
Härtung, T., Hoffmann, S., & Whaley, P. (2025) · Archives of Toxicology · doi:10.1007/s00204-025-03978-5 · Cited by 33
Pandey, D., Kumari, B., Singhal, N., & Kumar, M. (2020) · Scientific Reports · doi:10.1038/s41598-020-65981-3 · Cited by 24
Harrison, L., Kats, A., Kilroy, E., Butera, C. D., Jayashankar, A., Keleş, Ü., & Aziz‐Zadeh, L. (2021) · Scientific Reports · doi:10.1038/s41598-021-87455-w · Cited by 33
Kowarski, K. A., Delarue, J. J.-Y., Gaudet, B. J., & Martin, B. S. (2021) · JASA Express Letters · doi:10.1121/10.0004851 · Cited by 23
Product progression: a machine learning approach to forecasting industrial upgrading
Albora, G., Pietronero, L., Tacchella, A., & Zaccaria, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-28179-x · Cited by 28
Population temporal structure supplements the rate code during sensorimotor transformations
Jagadisan, U. K., & Gandhi, N. J. (2022) · Current Biology · doi:10.1016/j.cub.2022.01.015 · Cited by 25
Machine Learning Improves Upon Clinicians' Prediction of End Stage Kidney Disease
Chuah, A., Walters, G. D., Christiadi, D., Karpe, K. M., Kennard, A. L., Singer, R. F., Talaulikar, G. S., Ge, W., Suominen, H., Andrews, T. D., et al. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.837232 · Cited by 23
End-to-End Pathological Speech Detection Using Wavelet Scattering Network
Reddy, M. K., Keerthana, Y. M., & Alku, P. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3199669 · Cited by 24
“Five standard performance metrics, namely, recall, precision, F1-score, accuracy and Mathews correlation coefficient (MCC) are considered for evaluating the performance [5], [30].”
MethodHybrid intelligent predictive maintenance model for multiclass fault classification
Buabeng, A., Simons, A. J., Frempong, N. K., & Ziggah, Y. Y. (2023) · Soft Computing · doi:10.1007/s00500-023-08993-1 · Cited by 31
“7 Matthews Correlation Coefficient (MCC) 490 MCC (Equation (40)) measures the relationship between the observed and the predicted classification and is 491 generally regarded as a balanced metric for evaluating the performance of classifiers even with varying class 492 sizes (Boughorbel et al. 2017; Chicco 2017).”
BackgroundPredicting Microsleep States Using EEG Inter-Channel Relationships
Buriro, A. B., Shoorangiz, R., Weddell, S. J., & Jones, R. D. (2018) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2018.2878587 · Cited by 23
“These performance metrics were therefore combined into a single widely-used performance metric of phi for imbalanced biomedical data [51], [52] and was subsequently used to demonstrate the effect of feature preprocessing.”
MethodPrediction of cancer using customised fuzzy rough machine learning approaches
Chinnaswamy, A., & Ramakrishnan, S. (2018) · Healthcare Technology Letters · doi:10.1049/htl.2018.5055 · Cited by 23
Classification of Hyperspectral In Vivo Brain Tissue Based on Linear Unmixing
Cruz-Guerrero, I. A., León, R., Campos-Delgado, D. U., Ortega, S., Fabelo, H. A., & Callicó, G. M. (2020) · Applied Sciences · doi:10.3390/app10165686 · Cited by 27
“5 no better than a random prediction, and 0 total disagreement between prediction and observation [39].”
BackgroundMethodInfluentialPreinterventional Third-Molar Assessment Using Robust Machine Learning
Carvalho, J. S., Lotz, M., Rubi, L., Unger, S., Pfister, T., Buhmann, J. M., & Stadlinger, B. (2023) · Journal of Dental Research · doi:10.1177/00220345231200786 · Cited by 30
“For the evaluation of model performance, accuracy, precision, recall, Matthew’s correlation coefficient (MCC) (Chicco and Jurman 2020), and the F1-score were used, with the last 2 metrics being known to robustly mitigate imbalances in the label distribution (Boughorbel et al. 2017).”
MethodFatimah, B., Singhal, A., & Singh, P. (2024) · Medical Engineering & Physics · doi:10.1016/j.medengphy.2024.104102 · Cited by 22
Hillman, S. C., Wallace, L. O., Reinke, K. J., Hally, B., Jones, S., & Saldias, D. S. M. (2019) · Remote Sensing · doi:10.3390/rs11182118 · Cited by 22
“This measure has been used previously in the application of point cloud analysis [52,53] to compare observed and predicted binary classifications where the classes are imbalanced [54].”
MethodHalder, A. K., Giri, A. K., & Cordeiro, M. N. D. S. (2019) · Molecules · doi:10.3390/molecules24213909 · Cited by 24
Lithography Hotspot Detection with FFT-based Feature Extraction and Imbalanced Learning Rate
He, X., Deng, Y., Zhou, S., Li, R., Wang, Y., & Guo, Y. (2019) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3372044 · Cited by 22
“We add the metric MCC, since it is generally regarded as a balanced measure that can be used even if the classes are of very different sizes [3].”
Anatomical Modeling of Brain Vasculature in Two-Photon Microscopy by Generalizable Deep Learning
Tahir, W., Kura, S., Zhu, J., Cheng, X., Damseh, R., Tadesse, F., Seibel, A. J., Lee, B. S., Lesage, F., Sakadžić, S., et al. (2020) · BME Frontiers · doi:10.34133/2021/8620932 · Cited by 35
Test case prioritization using test case diversification and fault-proneness estimations
Mahdieh, M., Mirian‐Hosseinabadi, S., & Mahdieh, M. (2022) · Automated Software Engineering · doi:10.1007/s10515-022-00344-y · Cited by 38
“The very low p -values indicate that significance is confident.”
BackgroundTiwari, A., Liaqat, S., Liaqat, D., Gabel, M., de Lara, E., & Falk, T. H. (2021) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc46164.2021.9629949 · Cited by 22
“robust metric for imbalanced data [21].”
Cancer Detection and Prediction Using Genetic Algorithms
Bhandari, A., Tripathy, B. K., Jawad, K., Bhatia, S., Rahmani, M. K. I., & Mashat, A. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/1871841 · Cited by 26
“Matthews correlation coefficient (MCC) [77] and area under ROC curve (AUC) were also used for comparison: while AdaBoost-GA had the highest AUC value for all datasets, the results with MCC were not as consistent and other classifiers outperformed.”
MethodRetractedCasal-Guisande, M., Torres‐Durán, M., Mosteiro, M., Cerqueiro-Pequeño, J., Bouza-Rodríguez, J.-B., Fernández‐Villar, A., & Comesaña-Campos, A. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20043627 · Cited by 26
A dynamic trust management model for vehicular ad hoc networks
Aslan, M., & Şen, S. (2023) · Vehicular Communications · doi:10.1016/j.vehcom.2023.100608 · Cited by 35
Machine learning algorithms to predict risk of postoperative pneumonia in elderly with hip fracture
Guo, J., He, Q., Peng, C., Dai, R., Li, W., Su, Z., & Li, Y. (2023) · Journal of Orthopaedic Surgery and Research · doi:10.1186/s13018-023-04049-0 · Cited by 23
Tyagi, P., Sharma, A., Semwal, R., Tiwary, U. S., & Varadwaj, P. K. (2023) · Journal of Biomolecular Structure and Dynamics · doi:10.1080/07391102.2023.2258415 · Cited by 22
“The correlation between predicted and actual observations is referred to as MCC. MCC is more useful when evaluating the performance of a classification model with an imbalanced test dataset; sometimes in such cases, the F1-score and accuracy can be extremely misleading (Boughorbel et al., 2017).”
Diniz, A. P. M., Ciarelli, P. M., Salles, E. O. T., & Côco, K. F. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121963 · Cited by 26
Performance of In Silico Models for Mutagenicity Prediction of Food Contact Materials
Bossuyt, M. V., Hoeck, E. V., Raitano, G., Vanhaecke, T., Benfenati, E., Mertens, B., & Rogiers, V. (2018) · Toxicological Sciences · doi:10.1093/toxsci/kfy057 · Cited by 22
“This metric is compatible with imbalanced data and represents a widely used performance measure in biomedical research (Boughorbel et al., 2017).”
BackgroundMethodWu, X., Zhang, Q., & Wang, Y. (2019) · Spectrochimica Acta Part A Molecular and Biomolecular Spectroscopy · doi:10.1016/j.saa.2019.01.008 · Cited by 22
An approach for bug localization in models using two levels: model and metamodel
Arcega, L., Font, J., Haugen, Ø., & Cetina, C. (2019) · Software & Systems Modeling · doi:10.1007/s10270-019-00727-y · Cited by 24
“MCC is a balanced measure which can be used even if the search space and the predicted solution are of very different sizes [9].”
BackgroundBaltussen, E. J. M., Sterenborg, H. J. C. M., Ruers, T. J. M., & Dashtbozorg, B. (2019) · Biomedical Optics Express · doi:10.1364/boe.10.006096 · Cited by 22
A novel improved crow‐search algorithm to classify the severity in digital mammograms
Chakravarthy, S. R. S., & Rajaguru, H. (2020) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.22493 · Cited by 23
Catch and trade bans for seahorses can be negated by non‐selective fisheries
Vaidyanathan, T., Zhang, X., Balakrishnan, R., & Vincent, A. C. J. (2020) · Aquatic Conservation Marine and Freshwater Ecosystems · doi:10.1002/aqc.3419 · Cited by 24
Long-term implicit memory for sequential auditory patterns in humansPreprint
Bianco, R., Harrison, P. M. C., Hu, M., Bolger, C., Picken, S., Pearce, M. T., & Chait, M. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.02.14.949404 · Cited by 24
Exploring modeling techniques for predicting band gaps of Doped-ZnO: A Machine learning approach
Lamouadene, H., Kassaoui, M. E., Yadari, M. E., Kenz, A. E., & Benyoussef, A. (2025) · Chemical Physics · doi:10.1016/j.chemphys.2025.112603 · Cited by 26
Automated Burned Scar Mapping Using Sentinel-2 Imagery
Stavrakoudis, D. G., Katagis, T., Minakou, C., & Gitas, I. Z. (2020) · Journal of Geographic Information System · doi:10.4236/jgis.2020.123014 · Cited by 27
“Finally, MCC takes into consideration both omission and commission errors, being a correlation coefficient between the observed and predicted binary classifications and is generally regarded as a balanced measure that can be used even if the classes are of very different sizes [40].”
AKT Inhibitors: The Road Ahead to Computational Modeling-Guided Discovery
Halder, A. K., & Cordeiro, M. N. D. S. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms22083944 · Cited by 22
“To do so, statistical parameters such as the sensitivity, specificity, F-measure, accuracy and the Matthews correlation coefficient [41,42] values were carefully examined not only for the sub-training subset but also, to infer their external predictivity, firstly for the test set (n = 1160) and finally for the validat…”
BackgroundMethodInfluentialJong, L.-J. S., de Kruif, N., Geldof, F., Veluponnar, D., Sanders, J., Peeters, M.-J. T. F. D. V., van Duijnhoven, F. H., Sterenborg, H. J. C. M., Dashtbozorg, B., & Ruers, T. J. M. (2022) · Biomedical Optics Express · doi:10.1364/boe.455208 · Cited by 25
Huang, Y., & Mushi, R. (2022) · IEEE Access · doi:10.1109/access.2022.3205591 · Cited by 23
“Moreover, the worst classification results were observed for the CHROMA + LPC + MELSPEC feature set, with the F 1 score being 93.37% and the MCC being 0.863.”
BackgroundInfluentialJia, J., Wu, G., Li, M., & Qiu, W. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-05001-5 · Cited by 23
“When the distributions of samples are very imbalanced, MCC was considered to be the more noteworthy measure because it can more accurately reflect the quality of the model [43].”
BackgroundEscorcia-Rodríguez, J. M., Gaytan-Nuñez, E., Hernandez-Benitez, E. M., Zorro-Aranda, A., Tello‐Palencia, M. A., & Freyre-González, J. A. (2023) · Frontiers in Genetics · doi:10.3389/fgene.2023.1143382 · Cited by 24
“For the assessment of predictions not providing a score for each interaction, we used the MCC which is the best-suited coefficient for imbalanced datasets (Boughorbel et al., 2017).”
MethodCasal-Guisande, M., Álvarez-Pazó, A., Cerqueiro-Pequeño, J., Bouza-Rodríguez, J.-B., Lourido, G. P., & Comesaña-Campos, A. (2023) · Cancers · doi:10.3390/cancers15061711 · Cited by 28
Multilayer network alignment based on topological assessment via embeddings
Cinaglia, P., Milano, M., & Cannataro, M. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05508-5 · Cited by 21
“It is equiv - alent to chi-square statistics for a 2 x 2 contingency table [31].”
MethodImproving the drug discovery process by using multiple classifier systems
Ruano-Ordás, D., Yevseyeva, I., Basto-Fernandes, V., Méndez, J. R., & Emmerich, M. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.12.032 · Cited by 24
Heinrich, F., Wutke, M., Das, P. P., Kamp, M., Gültaş, M., Link, W., & Schmitt, A. O. (2020) · Genes · doi:10.3390/genes11060614 · Cited by 24
“From these measures, we calculated Accuracy (ACC), Sensitivity (true positive rate), Specificity (true negative rate), and the Matthews Correlation Coefficient (MCC) as below [33,34,63]:”
MethodGulino, M.-S., Gangi, L. D., Sortino, A., & Vangi, D. (2020) · Accident Analysis & Prevention · doi:10.1016/j.aap.2020.105864 · Cited by 23
Chongomweru, H., & Kasem, A. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05570-7 · Cited by 21
Shallow or Deep? An Empirical Study on Detecting Vulnerabilities using Deep Learning
Mazuera-Rozo, A., Mojica-Hanke, A. I., Linares‐Vásquez, M., & Bavota, G. (2021) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc52881.2021.00034 · Cited by 28
Zimbelman, E. G., & Keefe, R. F. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250624 · Cited by 23
Vanderwolf, K. J., Campbell, L. J., Taylor, D. R., Goldberg, T. L., Blehert, D. S., & Lorch, J. M. (2021) · Microbiology Spectrum · doi:10.1128/spectrum.00254-21 · Cited by 19
“The eastern data set had unequal numbers of the two susceptibility groups, but the Matthews correlation coefficient can be used even if classes are of very different sizes (37).”
BackgroundWeiner, L., Guidi, A., Doignon‐Camus, N., Giersch, A., Bertschy, G., & Vanello, N. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01535-z · Cited by 22
“In fact, it can be used in the case of unbalanced data, that is, when the number of subjects in the two groups are different, whereas accuracy can be biased by the results in the group with more subjects [40].”
BackgroundMiholca, D.-L., Tomescu, V.-I., & Czibula, G. (2022) · IEEE Access · doi:10.1109/access.2022.3181995 · Cited by 21
“8) Matthews Correlation Coefficient (MCC) [80] is also considered to be a good evaluation metric for imbalanced data sets and is computed as MCC = TP×TN−FP×FN √ (TP+FP)(TP+FN )(TN+FP)(TN+FN ) .”
InfluentialNetwork intrusion detection system by learning jointly from tabular and text‐based features
Düzgün, B., Çayır, A., Ünal, U., & Dağ, H. (2023) · Expert Systems · doi:10.1111/exsy.13518 · Cited by 30
“In imbalanced data problems, accuracy can be misleading, as it only measures the proportion of correct predictions and does not consider the data distribution (Equation 1) (Boughorbel et al., 2017).”
BackgroundMachine Learning for Prioritization of Thermostabilizing Mutations for G-Protein Coupled Receptors
Muk, S., Ghosh, S., Achuthan, S., Chen, X., Yao, X.-J., Sandhu, M., Griffor, M. C., Fennell, K. F., Che, Y., Shanmugasundaram, V., et al. (2019) · Biophysical Journal · doi:10.1016/j.bpj.2019.10.023 · Cited by 19
Neural predictive monitoring and a comparison of frequentist and Bayesian approaches
Bortolussi, L., Cairoli, F., Paoletti, N., Smolka, S. A., & Stoller, S. D. (2021) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-021-00623-1 · Cited by 20
First multi-target QSAR model for predicting the cytotoxicity of acrylic acid-based dental monomers
Halder, A. K., Delgado, A. H. S., & Cordeiro, M. N. D. S. (2021) · Dental Materials · doi:10.1016/j.dental.2021.12.014 · Cited by 21
Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph RepresentationPreprint
Zhuang, Y., Suneja, S., Thost, V., Domeniconi, G., Morari, A., & Laredo, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03341 · Cited by 37
Ge, F., Hu, J. S., Zhu, Y., Arif, M., & Yu, D. (2020) · Combinatorial Chemistry & High Throughput Screening · doi:10.2174/1386207323666201204140438 · Cited by 19
Predicting Blood Donors Using Machine Learning Techniques
Kauten, C., Gupta, A., Qin, X., & Richey, G. (2021) · Information Systems Frontiers · doi:10.1007/s10796-021-10149-1 · Cited by 21
Analysis of Machine Learning Methods for COVID-19 Detection Using Serum Raman Spectroscopy
Chen, D. (2021) · Applied Artificial Intelligence · doi:10.1080/08839514.2021.1975379 · Cited by 25
“The MCC metric accounts for true and false positives and negatives and is generally robust to class imbalance (Boughorbel et al. 2017).”
BackgroundUnknown radar waveform recognition system via triplet convolution network and support vector machine
Liu, L., & Li, X. (2022) · Digital Signal Processing · doi:10.1016/j.dsp.2022.103439 · Cited by 20
Anatomical Modeling of Brain Vasculature in Two-Photon Microscopy by Generalizable Deep Learning
Tahir, W., Kura, S., Zhu, J., Cheng, X., Damseh, R., Tadesse, F., Seibel, A. J., Lee, B. S., Lesage, F., Sakadžić, S., et al. (2020) · BME Frontiers · doi:10.34133/2020/8620932 · Cited by 19
Çinar, U., Çetin-Atalay, R., & Çetin, Y. Y. (2023) · Journal of Imaging · doi:10.3390/jimaging9020025 · Cited by 22
“Moreover, we have used the Matthews Correlation Coefficient (MCC) metric, which is generally suggested for the classifiers focusing on class imbalance problems in medical studies [37] as formulated in Equation (9).”
MethodXu, Z., Shen, X., & Cao, L. (2023) · Remote Sensing · doi:10.3390/rs15082144 · Cited by 19
Casal-Guisande, M., Cerqueiro-Pequeño, J., Bouza-Rodríguez, J.-B., & Comesaña-Campos, A. (2023) · Mathematics · doi:10.3390/math11112469 · Cited by 23
Software Bug Prediction Using Reward-Based Weighted Majority Voting Ensemble Technique
Kumar, R., & Chaturvedi, A. (2023) · IEEE Transactions on Reliability · doi:10.1109/tr.2023.3295598 · Cited by 23
Reiter, T., & Schoedel, R. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02252-9 · Cited by 35
“The AUC metric can be considered robust to class imbalance (Boughorbel et al., 2017).”
BackgroundTest Input Prioritization for Machine Learning Classifiers
Dang, X., Li, Y., Papadakis, M., Klein, J., Bissyandé, T. F., & Traon, Y. L. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3350019 · Cited by 29
“Their approach consists of two distinct models: a statistics-based model and a probability-based model, both of which quantify the fault detection capability of each test case.”
HyperPCM: Robust Task-Conditioned Modeling of Drug–Target Interactions
Svensson, E., Hoedt, P.-J., Hochreiter, S., & Klambauer, G. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01417 · Cited by 25
Awareness, Navigation, and Use of Feed Control Settings Online
Hsu, S., Vaccaro, K., Yue, Y., Rickman, A., & Karahalios, K. G. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376583 · Cited by 34
Metadata
- Authors
- Boughorbel, S., Jarray, F., & El-Anbari, M.
- Affiliations
- Sidra Medical and Research Center (QA)
- Conservatoire National des Arts et Métiers (FR)
- Corresponding author
- Boughorbel, Sabri
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2017-06-02
- Volume
- 12
- Issue
- 6
- Pages
- e0177678
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5456046
- OpenAlex
- W2620760558
- Cited by
- 1,222
- References
- 29
- Citations by year
- 2026: 90 · 2025: 104 · 2024: 111 · 2023: 163 · 2022: 179 · 2021: 224 · 2020: 165 · 2019: 120 · 2018: 64 · 2017: 2
- FWCI
- 57.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Qatar Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Keywords
- class imbalance, Matthews correlation coefficient, Fréchet derivative, binary classification, biomedical classification
- MeSH terms
- Algorithms, Bayes Theorem, Models, Theoretical, Computational Biology, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.