Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Data imbalance is frequently encountered in biomedical applications. Resampling techniques can be used in binary classification to tackle this issue. However such solutions are not desired when the number of samples in the small class is limited. Moreover the use of inadequate performance metrics, such as accuracy, lead to poor generalization results because the classifiers tend to predict the largest size class. One of the good approaches to deal with this issue is to optimize performance metrics that are designed to handle data imbalance. Matthews Correlation Coefficient (MCC) is widely used in Bioinformatics as a performance metric. We are interested in developing a new classifier based on the MCC metric to handle imbalanced data. We derive an optimal Bayes classifier for the MCC metric using an approach based on Frechet derivative. We show that the proposed algorithm has the nice theoretical property of consistency. Using simulated data, we verify the correctness of our optimality result by searching in the space of all possible binary classifiers. The proposed classifier is evaluated on 64 datasets from a wide range data imbalance. We compare both classification performance and CPU efficiency for three classifiers: 1) the proposed algorithm (MCC-classifier), the Bayes classifier with a default threshold (MCC-base) and imbalanced SVM (SVM-imba). The experimental evaluation shows that MCC-classifier has a close performance to SVM-imba while being simpler and more efficient.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“since it has been shown to present a balanced score when dealing with imbalanced data [11, 6] as is our case with fewer manipulated images.”
From BioFors: A Large Biomedical Image Forensics Dataset (2021) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399
A Probabilistic Theory of Pattern Recognition
Devroye, L., Györfi, L., & Lugosi, G. (1996) · Stochastic Modelling and Applied Probability · doi:10.1007/978-1-4612-0711-5 · Cited by 3,274
Assessing the accuracy of prediction algorithms for classification: an overview
Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A. F., & Nielsen, H. (2000) · Bioinformatics · doi:10.1093/bioinformatics/16.5.412 · Cited by 2,111
A sequential algorithm for training text classifiers
Lewis, D., & Gale, W. A. (1994) · arXiv (Cornell University) · doi:10.5555/188490.188495 · Cited by 1,731
A Sequential Algorithm for Training Text Classifiers
Lewis, D. D., & Gale, W. A. (1994) · SIGIR ’94 · doi:10.1007/978-1-4471-2099-5_1 · Cited by 1,208
Applying Support Vector Machines to Imbalanced Datasets
Akbani, R., Kwek, S., & Japkowicz, N. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30115-8_7 · Cited by 1,134
Yang, J., Roy, A., & Zhang, Y. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt447 · Cited by 1,035
MAQC Consortium (2010) · Nature Biotechnology · doi:10.1038/nbt.1665 · Cited by 889
Imbalanced Learning: Foundations, Algorithms, and Applications
(2013) · Wiley eBooks · doi:10.1002/9781118646106 · Cited by 831
Cited by
Items 201–300 of 1,000
Machine Learning for Digital Twins to Predict Responsiveness of Cyber-Physical Energy Systems
Snijders, R., Pileggi, P., Broekhuijsen, J., Verriet, J., Wiering, M., & Kok, K. (2020) · Workshop on Modeling and Simulation of Cyber-Physical Energy Systems (MSCPES) · doi:10.1109/mscpes49613.2020.9133695 · Cited by 42
“It has been shown to be robust against unbalanced classes [23].”
BackgroundPandey, S. K., Rathee, D., & Tripathi, A. K. (2020) · IET Software · doi:10.1049/iet-sen.2020.0119 · Cited by 38
“MCC is high when the prediction is unbiased [95, 96] towards any class, or low otherwise.”
BackgroundPerinatal Risk Factors and Outcome Coding in Clinical and Administrative Databases
Tawfik, D. S., Gould, J. B., & Profit, J. (2019) · PEDIATRICS · doi:10.1542/peds.2018-1487 · Cited by 40
O’Neil, G. L., Saby, L., Band, L. E., & Goodall, J. L. (2019) · Water Resources Research · doi:10.1029/2019wr024784 · Cited by 43
“4352 MCCmetric has been shown to be suitable for imbalanced scenarios (e.g., Boughorbel et al., 2017); however, its calculation includes the number of true negative samples.”
BackgroundRobust Inhibition-Augmented Operator for Delineation of Curvilinear Structures
Strisciuglio, N., Azzopardi, G., & Petkov, N. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2922096 · Cited by 40
“We compute the Matthews correlation coefficient (MCC), which is a suitable measurement of the accuracy of binary classification in the case of unbalanced classes [57].”
MethodMomenzadeh, M., Sehhati, M., & Rabbani, H. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103570 · Cited by 43
Dreizin, D., Goldmann, F., LeBedis, C. A., Boscak, A. R., Dattwyler, M. P., Bodanapally, U. K., Li, G., Anderson, S. W., Maier, A., & Unberath, M. (2021) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-020-00399-x · Cited by 38
“The algorithm determines whether there is rotational or translational instability by application of an individual threshold anchored to the Matthews correlation coefficient [55, 56].”
MethodSingh, N., Bapi, R. S., & Vinod, P. K. U. (2018) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2018.06.030 · Cited by 41
Deep Learning Approaches for Predictive Masquerade Detection
Elmasry, W., Akbulut, A., & Zaim, A. H. (2018) · Security and Communication Networks · doi:10.1155/2018/9327215 · Cited by 37
“(xii) Matthews Correlation Coefficient (MCC) is a performance metric that takes into account true and false positives and negatives and is generally regarded as a balanced measure which can be used even if the classes are of very different sizes (imbalance dataset) [68].”
BackgroundNascimento, M. H. C., Marcarini, W. D., Folli, G. S., Filho, W. G. D. S., Barbosa, L. L., de Paulo, E. H., Vassallo, P. F., Mill, J. G., Baraúna, V. G., Martin, F. L., et al. (2022) · Analytical Chemistry · doi:10.1021/acs.analchem.1c04162 · Cited by 43
Enhanced Anomaly-Based Fault Detection System in Electrical Power Grids
Elmasry, W., & Wadi, M. (2022) · International Transactions on Electrical Energy Systems · doi:10.1155/2022/1870136 · Cited by 47
Liu, Y., Ли, Б., Yang, S., & Li, Z. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121694 · Cited by 44
Viral Immunogenicity Prediction by Machine Learning Methods
Doneva, N., & Dimitrov, I. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25052949 · Cited by 40
“Information gain is a measure of the reduction in entropy or uncertainty provided by a particular attribute.”
BackgroundMachine learning in predictive biocatalysis: A comparative review of methods and applications
Tripathi, N., Hérisson, J., & Faulon, J. (2025) · Biotechnology Advances · doi:10.1016/j.biotechadv.2025.108698 · Cited by 40
Identification of infectious disease-associated host genes using machine learning techniques
Barman, R. K., Mukhopadhyay, A., Maulik, U., & Das, S. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3317-0 · Cited by 39
Lovrić, M., Đuričić, T., Tran, H., Hussain, H., Lacić, E., Rasmussen, M. A., & Kern, R. (2021) · Pharmaceuticals · doi:10.3390/ph14080758 · Cited by 49
Calado, A., Errico, V., & Saggio, G. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3109732 · Cited by 40
Ge, F., Zhang, Y., Xu, J., Arif, M., Song, J., & Yu, D. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab530 · Cited by 39
Dual Inhibitors of AChE and BACE-1 for Reducing Aβ in Alzheimer’s Disease: From In Silico to In Vivo
Stern, N., Gacs, A., Tátrai, E., Flachner, B., Hajdú, I., Dobi, K., Bágyi, I., Dormán, G., Lőrincz, Z., Cseh, S., et al. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms232113098 · Cited by 35
A comparison between TLS and UAS LiDAR to represent eucalypt crown fuel characteristics
Hillman, S. C., Wallace, L. O., Reinke, K. J., & Jones, S. (2021) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2021.09.008 · Cited by 40
A novel weighted TPR-TNR measure to assess performance of the classifiers
Jadhav, A. S. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113391 · Cited by 42
Post, C., Rietz, C., Büscher, W., & Müller, U. (2020) · Sensors · doi:10.3390/s20143863 · Cited by 44
“MCC was chosen because it takes into account all parts of the contingency table and thus is suited and recommended for imbalanced classification problems [20].”
MethodFunk, C. C., Casella, A. M., Jung, S., Richards, M. A., Rodriguez, A., Shannon, P., Donovan-Maiye, R., Heavner, B., Chard, K., Xiao, Y., et al. (2020) · Cell Reports · doi:10.1016/j.celrep.2020.108029 · Cited by 35
Avoiding Occupancy Detection From Smart Meter Using Adversarial Machine Learning
Yilmaz, I., & Siraj, A. (2021) · IEEE Access · doi:10.1109/access.2021.3057525 · Cited by 41
“These metrics are reliable and robust parameters in the presence of class imbalance so that they are commonly used for evaluating the classification of a highly imbalanced datasets [78].”
BackgroundEchoBERT: A Transformer-Based Approach for Behavior Detection in Echograms
Måløy, H. (2020) · IEEE Access · doi:10.1109/access.2020.3042337 · Cited by 35
“We also report our results using the Matthews Correlation Coefficient (MCC) score [14], which has been shown to be robust to imbalanced datasets [1].”
ResultBuabeng, A., Simons, A. J., Frempong, N. K., & Ziggah, Y. Y. (2021) · SN Applied Sciences · doi:10.1007/s42452-021-04598-1 · Cited by 38
“MCC is generally known to be a balanced metric for assessing classifiers with varying class sizes [108].”
BackgroundBiometric-Based Human Recognition Systems: An Overview
Palma, D., & Montessoro, P. L. (2022) · IntechOpen eBooks · doi:10.5772/intechopen.101686 · Cited by 44
Prediction of anti-inflammatory peptides by a sequence-based stacking ensemble model named AIPStack
Deng, H., Lou, C., Wu, Z., Li, W., Liu, G., & Tang, Y. (2022) · iScience · doi:10.1016/j.isci.2022.104967 · Cited by 37
“For imbalanced datasets, MCC and AUC usually perform better in evaluating the model performance than other metrics (Boughorbel et al., 2017).”
Aristizabal, F., Chegini, T., Petrochenkov, G., Salas, F. R., & Judge, J. (2024) · Hydrology and earth system sciences · doi:10.5194/hess-28-1287-2024 · Cited by 44
“However, it does value both conditions (inundated and not inundated) to have equal impact (Chicco and Jurman, 2020; Chicco et al., 2021a, b; Boughorbel et al., 2017).”
BackgroundMethodHamerla, G., Meyer, H., Hambsch, P. K., Wolf, U., Kuhnt, T., Hoffmann, K., & Surov, A. (2019) · Cancers · doi:10.3390/cancers11111680 · Cited by 37
“In these cases, classification methods tend to be biased towards the majority class and the use of imbalance-correcting performance measures is recommended [20,21].”
BackgroundHalder, A. K., Melo, A., & Cordeiro, M. N. D. S. (2019) · Chemosphere · doi:10.1016/j.chemosphere.2019.125489 · Cited by 37
Machine-learning dessins d’enfants: explorations via modular and Seiberg–Witten curves
He, Y., Hirst, E., & Peterken, T. (2020) · Journal of Physics A Mathematical and Theoretical · doi:10.1088/1751-8121/abbc4f · Cited by 45
“for {tp, tn, f p, fn} representing {true positive, true negative, false positive, false negative} respectively [53].”
BackgroundKrauss, E., Haberer, J., Barreto, G., Degen, M., Seeger, W., & Günther, A. (2020) · Journal of Breath Research · doi:10.1088/1752-7163/ab8c50 · Cited by 36
“In essence, the MCC is a correlation between the observed and predicted binary classifications where a value of+1 represents a perfect condition, 0 no better than randomprediction and −1 indicates total disagreement between prediction and observation [20].”
BackgroundEstimating tie strength in social networks using temporal communication data
Ureña-Carrión, J., Saramäki, J., & Kivelä, M. (2020) · EPJ Data Science · doi:10.1140/epjds/s13688-020-00256-5 · Cited by 37
Encoder-decoder based convolutional neural networks for image forgery detection
Biach, F. Z. E., Iala, I., Laanaya, H., & Minaoui, K. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10158-3 · Cited by 46
“These methods are provided in an accessible Matlab toolbox written by Zamboglou et al. [13].”
BackgroundMethodDetection of Faults in Electrical Power Grids Using an Enhanced Anomaly-Based Method
Elmasry, W., & Wadi, M. (2022) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-022-07030-x · Cited by 40
“In this paper, eight well-known evaluation measures are utilized to assess the performance of the OC-SVM and PCA models, namely, accuracy, precision, recall, F1-score, False Alarm Rate (FAR), Specificity, False Negative Rate (FNR), Matthews Correlation Coeffi-cient (MCC) [48,49].”
MethodPredicting entrepreneurial activity using machine learning
Schade, P., & Schuhmacher, M. C. (2022) · Journal of Business Venturing Insights · doi:10.1016/j.jbvi.2022.e00357 · Cited by 45
Seizure detection devices for use in antiseizure medication clinical trials: A systematic review
Kurada, A. V., Srinivasan, T., Hammond, S., Ulate-Campos, A., & Bidwell, J. (2019) · Seizure · doi:10.1016/j.seizure.2019.02.007 · Cited by 37
Siebring, J., Valente, J. R. P., Franceschini, M. H. D., Kamp, J., & Kooistra, L. (2019) · Sensors · doi:10.3390/s19245477 · Cited by 33
“This metric measures the quality of binary classification and was selected for its ability to account for varying class sample sizes [30].”
BackgroundComparative study of quality estimation of binary classification
Старовойтов, В. В., & Golub, Y. I. (2020) · Informatics · doi:10.37661/1816-0301-2020-17-1-87-101 · Cited by 73
“В статье [16] показано, что для двух классов, случайно сгенерированных и несбалансированных, функции MCC и AUC достаточно устойчивы.”
Nawabi, J., Kniep, H. C., Kabiri, R., Broocks, G., Faizy, T. D., Thaler, C. P., Schön, G., Fiehler, J., & Hanning, U. M. P. (2020) · Frontiers in Neurology · doi:10.3389/fneur.2020.00285 · Cited by 34
Classification of pathological disorders in children using random forest algorithm
Bebortta, S., Panda, M., & Panda, S. (2020) · International Conference on Emerging Trends in Information Technology and Engineering (ICETITE) · doi:10.1109/ic-etite47903.2020.253 · Cited by 37
“The Matthews Correlation Coefficient (MCC) is one of the most popularly used techniques for handling imbalanced datasets and enhancing the accuracies of prediction models [24], [25].”
MethodPérez, E., & Ventura, S. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06655-7 · Cited by 37
“MCC is widely used in Bioinformatics as a performance metric [78], and it is specially designed to analyze the predictive performance on unbalanced data.”
MethodGe, F., Zhu, Y., Xu, J., Arif, M., Song, J., & Yu, D. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.11.024 · Cited by 34
“Among all performance metrics, MCC is considered to be the best indicator for evaluating the model performance, especially on unbalanced datasets [55].”
BackgroundSVDNVLDA: predicting lncRNA-disease associations by Singular Value Decomposition and node2vec
Li, J., Li, J., Kong, M., Wang, D., Fu, K., & Shi, J. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04457-1 · Cited by 35
“The evaluation metrics used in classifier selection and parameter adjustment processes contained Accuracy ( Acc ), Sensitivity ( Sen ), Specificity ( Spec ), Precision ( Prec ), and Matthews correlation coefficient ( MCC ) [32, 33].”
MethodPérez, E., & Ventura, S. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102556 · Cited by 37
TCR-ESM: Employing protein language embeddings to predict TCR-peptide-MHC binding
Yadav, S., Vora, D. S., Sundar, D., & Dhanjal, J. K. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.11.037 · Cited by 41
“The MCC results in a high score only when the predictions are reliable on all of the four categories-true and false positives as well as true and false negatives, while being proportional to the size of positive and negative samples in the dataset [31].”
BackgroundCuesta–Frau, D., Dakappa, P. H., Mahabala, C., & Gupta, A. R. (2020) · Entropy · doi:10.3390/e22091034 · Cited by 32
“It is then where MCC exhibits its superior robustness and reliability [28].”
BackgroundMethodLee, G., Jang, G. H., Kang, H. Y., & Song, G. (2021) · PLoS ONE · doi:10.1371/journal.pone.0253760 · Cited by 39
Multilayer perceptron artificial neural networks-based model for credit card fraud detection
Kasasbeh, B., Aldabaybah, B., & Ahmad, H. (2022) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v26.i1.pp362-373 · Cited by 32
Deep Learning and Vision-Based Early Drowning Detection
Shatnawi, M., Albreiki, F., Alkhoori, A., & Alhebshi, M. (2023) · Information · doi:10.3390/info14010052 · Cited by 32
Sctensor detects many-to-many cell–cell interactions from single cell RNA-sequencing data
Tsuyuzaki, K., Ishii, M., & Nikaido, I. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05490-y · Cited by 80
Yan, S., Yao, X., Sun, J., Huang, W., Yang, L., Zhang, C., Gao, B., Yang, J., Yun, W., & Zhu, D. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.108902 · Cited by 31
Harnessing machine learning to predict cytochrome P450 inhibition through molecular properties
Zahid, H., Tayara, H., & Chong, K. T. (2024) · Archives of Toxicology · doi:10.1007/s00204-024-03756-9 · Cited by 32
“We considered Mathews correlation coefficient (MCC) and balanced accuracy (BA) as our main evaluation measures because of the imbalanced dataset (Thölke et al. 2023; Boughorbel et al. 2017); however, sensitivity, specificity, and precision are also calculated for all five models.”
A Comparative Study of Machine Learning Algorithms and Their Ensembles for Botnet Detection
Ryu, S., & Yang, B. (2018) · Journal of Computer and Communications · doi:10.4236/jcc.2018.65010 · Cited by 32
“According to [22], MCC is more robust to an imbalanced data where classification methods tend to biased toward the majority class than the F1 score or accuracy.”
BackgroundAutoregressive Planet Search: Methodology
Caceres, G. A., Feigelson, E. D., Babu, G. J., Bahamonde, N., Christen, A., Bertin, K., Meza, C., & Curé, M. (2019) · The Astronomical Journal · doi:10.3847/1538-3881/ab26b8 · Cited by 39
“Matthew’s correlation coefficient, an arithmetic combination of True Positive and False Positive counts related to the χ(2) statistic for a 2× 2 contingency table, is particularly recommended when the populations of the classes are highly imbalanced (Boughorbel et al. 2017).”
MethodWallace, L. O., Bellman, C. J., Hally, B., Hernández, J., Jones, S., & Hillman, S. C. (2019) · Forests · doi:10.3390/f10030284 · Cited by 31
Rajbahadur, G. K., Wang, S., Kamei, Y., & Hassan, A. E. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2924371 · Cited by 42
“and Mathew’s Correlation Coefficient (MCC), since many prior studies studied the performance of classifiers using these measures [6], [8], [73].”
TTAgP 1.0: A computational tool for the specific prediction of tumor T cell antigens
Lissabet, J. F. B., Belén, L. H., & Farías, J. G. (2019) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2019.107103 · Cited by 34
Halder, A. K., & Cordeiro, M. N. D. S. (2019) · International Journal of Molecular Sciences · doi:10.3390/ijms20174191 · Cited by 31
“The classification results obtained for the sub-training and test sets are presented in Table 1, outlining the overall performance of the present mt-QSAR LDA model. a NDTotal—Number of total data-points; NDactive—Number of active data-points, NDinactive—Number of inactive data-points CCD—Correctly classified data, MCC…”
MethodResultInfluentialNELLY: Flow Detection Using Incremental Learning at the Server Side of SDN-Based Data Centers
Estrada‐Solano, F., Rendón, O. M. C., & da Fonseca, N. L. S. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2947291 · Cited by 33
“The BinNom headers were found to enable the best performance of the majority of the algorithms for the UNI1 and UNI2 datasets.”
BackgroundInfluentialSequence-Based Prediction of Metamorphic Behavior in Proteins
Chen, N., Das, M., LiWang, A., & Wang, L. (2020) · Biophysical Journal · doi:10.1016/j.bpj.2020.07.034 · Cited by 33
“For each combination of our parameters CR and DIthre, the MCC is computed from a matrix of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), called a confusion matrix (35):”
MethodHuang, N., Chou, D., Lee, C., Wu, F., Chuang, A.-C., Chen, Y., & Tsai, Y. (2020) · IET Smart Cities · doi:10.1049/iet-smc.2020.0068 · Cited by 31
“According to the results of our experiments, the FPR was 0.0441 when the testing data included 7000 images each of good and bad coffee beans.”
BackgroundInfluentialAnteghini, M., dos Santos, V. A. P. M., & Saccenti, E. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms22126409 · Cited by 31
“The MCC is appropriate also in presence of class unbalance [54].”
BackgroundSaygılı, A. (2021) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-021-06240-z · Cited by 33
Data mining analyses for precision medicine in acromegaly: a proof of concept
Gil, J., Marqués-Pamies, M., Sampedro, M., Webb, S. M., Serra, G., Salinas, I., Blanco, A., Valassi, E., Carrato, C., Pico, A. M., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-12955-2 · Cited by 33
Wang, Z., Niu, Y., Vashisth, T., Li, J., Madden, R., Livingston, T., & Wang, Y. (2022) · Horticulture Research · doi:10.1093/hr/uhac145 · Cited by 37
“MCC exemplifies the correlation between true and predicted labels [34]:”
BackgroundDeep Transfer Learning Enables Robust Prediction of Antimicrobial Resistance for Novel Antibiotics
Ren, Y., Chakraborty, T., Doijad, S. P., Falgenhauer, L., Falgenhauer, J. C., Goesmann, A., Schwengers, O., & Heider, D. (2022) · Antibiotics · doi:10.3390/antibiotics11111611 · Cited by 34
“It is the method of choice for imbalanced datasets [41]:”
MethodA systematic review on deep learning based methods for cervical cell image analysis
Fang, M., Liao, B., Lei, X., & Wu, F. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128630 · Cited by 46
An analytical threshold for combining Bayesian Networks
Gross, T. J., Bessani, M., Darwin, W., Araújo, R. B., Vale, F. A. C., & Maciel, C. D. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.03.014 · Cited by 32
Optimizing for Recall in Automatic Requirements Classification: An Empirical Study
Winkler, J., Grönberg, J., & Vogelsang, A. (2019) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2019.00016 · Cited by 33
“It can be used even if the classes are of very different sizes, a common property of software defect data [31], [32].”
BackgroundThe Sensorimotor Basis of Whisker-Guided Anteroposterior Object Localization in Head-Fixed Mice
Cheung, J. A., Maire, P., Kim, J., Sy, J., & Hires, S. A. (2019) · Current Biology · doi:10.1016/j.cub.2019.07.068 · Cited by 33
“MCC provides an unbiased metric of model performance in light of imbalanced datasets [57].”
Alignment-Free Method to Predict Enzyme Classes and Subclasses
Concu, R., & Cordeiro, M. N. D. S. (2019) · International Journal of Molecular Sciences · doi:10.3390/ijms20215389 · Cited by 30
“Standard statistics, such as the specificity (Sp), sensitivity (Sn), probability of error (p), cross-validation, and the Matthews correlation coefficient (MCC) [58], were used to assess the discriminatory power of the model.”
MethodInfluentialEhatisham-ul-Haq, M., Azam, M. A., Amin, Y., & Naeem, U. (2020) · IEEE Access · doi:10.1109/access.2020.2964237 · Cited by 34
“In addition to BAC, Mathews Correlation Coefficient (MCC) is used to evaluate the classifier performance, which is primarily considered as a balanced measure even if the classes are of different sizes [67], [68].”
MethodThe flood cooking book: ingredients and regional flavors of floods across Germany
Merz, R., Tarasova, L., & Basso, S. (2020) · Environmental Research Letters · doi:10.1088/1748-9326/abb9dd · Cited by 35
Zoppi, T., & Ceccarelli, A. (2021) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103106 · Cited by 34
Sharan, T. S., Tripathi, S., Sharma, S., & Sharma, N. (2021) · IETE Technical Review · doi:10.1080/02564602.2021.1955760 · Cited by 32
“Dice loss [49] is defined as: The segmented results were evaluated using well known metrics namely Jaccard Index [50,51], Dice score [52], Matthews correlation coefficient (MCC) [53], accuracy, specificity, recall, and precision (Figure 2).”
MethodSoftware evaluation for de novo detection of transposons
Rodriguez, M., & Makałowski, W. (2022) · Mobile DNA · doi:10.1186/s13100-022-00266-2 · Cited by 33
Neto, E. J. D. M., & Clementino, M. B. M. (2022) · PLoS ONE · doi:10.1371/journal.pone.0272287 · Cited by 56
“Following their work and others [27, 28], we have decided to use the Matthews Correlation Coefficient (MCC) as the primary way to evaluate our classifiers.”
MethodTaouktsis, X., & Zikopoulos, C. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122010 · Cited by 35
Xie, X., & Li, A. (2018) · Geoderma · doi:10.1016/j.geoderma.2018.03.029 · Cited by 34
Ismail, S. S., Dandan, S., Dawoud, D. W., & Reza, H. (2024) · IEEE Access · doi:10.1109/access.2024.3432454 · Cited by 33
“Micro F1-score, and Micro F1-score can be expressed as follows : In addition, we use MCC which provides a comprehensive measure of classification performance specifically when dealing with imbalanced dataset since it exhibited consistent performance across different classifiers [46].”
MethodMuscas, G., Matteuzzi, T., Becattini, E., Orlandini, S., Battista, F., Laiso, A., Nappini, S., Limbucci, N., Renieri, L., Carangelo, B. R., et al. (2020) · Acta Neurochirurgica · doi:10.1007/s00701-020-04484-6 · Cited by 34
“A score equal to 0 means the model makes no better prediction than a random guess [4].”
BackgroundMethodInfluentialMapping Coastal Dune Landscape through Spectral Rao’s Q Temporal Diversity
Marzialetti, F., Febbraro, M. D., Malavasi, M., Giulio, S., Acosta, A. T. R., & Carranza, M. L. (2020) · Remote Sensing · doi:10.3390/rs12142315 · Cited by 34
“This coefficient is widely used to evaluate the accuracy of classifications [86,90], as it proved reliable to evaluate the accuracy of classification when the classes had different sizes; its range of −1 to 1 and values close to 1 represent a perfect accuracy [87,91].”
MethodA Forest Monitoring System for Tanzania
John, E., Bunting, P., Hardy, A., Silayo, D. S., & Masunga, E. (2021) · Remote Sensing · doi:10.3390/rs13163081 · Cited by 28
“The accuracy metrics from the error matrix were summarised as both an overall accuracy (OA), user and producer accuracy (UA and PA), allocation disagreement (AD), quantity disagreement (QD) [35,36], F1-score [37], and the Matthews Correlation Coefficient (MCC) [38].”
MethodInfluentialA wrapper based binary bat algorithm with greedy crossover for attribute selection
Akila, S., & Christe, S. A. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115828 · Cited by 29
Mehrjerd, A., Rezaei, H., Eslami, S., Ratna, M. B., & Ghaebi, N. K. (2022) · Scientific Reports · doi:10.1038/s41598-022-10902-9 · Cited by 38
Interpretable Machine Learning Assessment
Han, H., Wu, Y., Wang, J., & Han, A. (2022) · Neurocomputing · doi:10.1016/j.neucom.2023.126891 · Cited by 35
Goodwin, T. R., & Demner‐Fushman, D. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa004 · Cited by 29
On the educated selection of unsupervised algorithms via attacks and anomaly classes
Zoppi, T., Ceccarelli, A., Salani, L., & Bondavalli, A. (2020) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2020.102474 · Cited by 29
A strong hybrid AdaBoost classification algorithm for speaker recognition
Karthikeyan, V., & Priyadharsini, S. S. (2021) · Sadhana · doi:10.1007/s12046-021-01649-6 · Cited by 28
“1}, for the truly correct classification MCC value is ?”
MethodInfluentialA semantic analysis-driven customer requirements mining method for product conceptual design
Wu, X., Hong, Z., Feng, Y., Li, M.-D., Lou, S., & Tan, J. (2022) · Scientific Reports · doi:10.1038/s41598-022-14396-3 · Cited by 39
“On the one hand, granular computing 27–29 and data resampling 30,31 are utilized to change the imbalance rate of training dataset.”
MethodArias, E., Sierra, V., Prado, N., González, P., Fiorentini, G., Díaz, J., & Oliván, M. (2022) · Foods · doi:10.3390/foods11233808 · Cited by 28
“An MCC value of 1 indicates perfect prediction, and−1 indicates total disagreement between the prediction and true values, and 0 means no better than random prediction [50].”
BackgroundWang, J., Liu, Y., & Tian, B. (2024) · Journal of Cheminformatics · doi:10.1186/s13321-024-00920-2 · Cited by 33
Naghavi, A., Teismann, T., Asgari, Z., Mohebbian, M. R., Mansourian, M., & Mañanas, M. Á. (2020) · Diagnostics · doi:10.3390/diagnostics10110956 · Cited by 34
“Alternative objective indices such as AUC and MCC are preferred [79,81].”
BackgroundKhalili, E., Ramazi, S., Ghanati, F., & Kouchaki, S. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac015 · Cited by 28
Bhattacharjee, S., Saha, B., Bhattacharyya, P., & Saha, S. (2022) · Journal of Computational Science · doi:10.1016/j.jocs.2022.101768 · Cited by 31
BOFRF: A Novel Boosting-Based Federated Random Forest Algorithm on Horizontally Partitioned Data
Gençtürk, M., Sınacı, A. A., & Çiçekli, N. K. (2022) · IEEE Access · doi:10.1109/access.2022.3202008 · Cited by 31
“MCC considers all values in the confusion matrix; hence, it is not affected by imbalanced or skewed datasets [42].”
BackgroundGuidelines for the prediction of species interactions through binary classification
Poisot, T. (2023) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14071 · Cited by 30
“…the use of Cohen's 𝜅 , again in favour of MCC, as the relative nature of 𝜅 means that a worse classifier can be picked over a better one; similarly, Boughorbel et al. (2017) recommend MCC over other measures of performance for imbalanced data, as it has more desirable statistical properties.”
MethodCombining Deep Learning and Multi-Source GIS Methods to Analyze Urban and Greening Changes
Francini, M., Salvo, C., & Vitale, A. (2023) · Sensors · doi:10.3390/s23083805 · Cited by 31
“Adam was used as the model optimizer [63] with an initial learning rate of 0.”
Method
Metadata
- Authors
- Boughorbel, S., Jarray, F., & El-Anbari, M.
- Affiliations
- Sidra Medical and Research Center (QA)
- Conservatoire National des Arts et Métiers (FR)
- Corresponding author
- Boughorbel, Sabri
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2017-06-02
- Volume
- 12
- Issue
- 6
- Pages
- e0177678
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5456046
- OpenAlex
- W2620760558
- Cited by
- 1,222
- References
- 29
- Citations by year
- 2026: 90 · 2025: 104 · 2024: 111 · 2023: 163 · 2022: 179 · 2021: 224 · 2020: 165 · 2019: 120 · 2018: 64 · 2017: 2
- FWCI
- 57.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Qatar Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Keywords
- class imbalance, Matthews correlation coefficient, Fréchet derivative, binary classification, biomedical classification
- MeSH terms
- Algorithms, Bayes Theorem, Models, Theoretical, Computational Biology, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.