A Comparison of MCC and CEN Error Measures in Multi-Class Prediction
Open accessAbstract
We show that the Confusion Entropy, a measure of performance in multiclass problems has a strong (monotone) relation with the multiclass generalization of a classical metric, the Matthews Correlation Coefficient. Analytical results are provided for the limit cases of general no-information (n-face dice rolling) of the binary classification. Computational evidence supports the claim in the general case.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
MCC is the only binary classification rate that generates a high score only if the binary predictor correctly predicts the majority of positive data instances and the majority of negative data instances.
“MCC is the only binary classification rate that generates a high score only if the binary predictor was able to correctly predict the majority of positive data instances and the majority of negative data instances [80, 97].”
From The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation (2020) · cites this paper as [80]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486 · see citing works
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 49,574 · see citing works
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · see citing works
The Proceedings of the National Academy of Sciences
Richards, J. W. (1915) · Science · doi:10.1126/science.42.1074.161 · Cited by 13,941 · see citing works
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 11,966 · see citing works
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515 · see citing works
The use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217 · see citing works
Repeated observation of breast tumor subtypes in independent gene expression data sets
Sørlie, T., Tibshirani, R. J., Parker, J. S., Hastie, T. J., Marron, J. S., Nobel, A. B., Deng, S., Johnsen, H., Pesich, R., Geisler, S. B., et al. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0932692100 · Cited by 5,428 · see citing works
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399 · cited 1× · see citing works
“This latter function was introduced in (Matthews, 1975) and it is also known as the φ -coe ffi cient, corresponding for a 2 × 2 contingency table to the square root of the average χ 2 statistic χ 2 / n . p MCC has recently attracted the attention of the machine learning community (Baldi et al., 2000)…”
BackgroundA Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems
Hand, D. J., & Till, R. J. (2001) · Machine Learning · doi:10.1023/a:1010920819831 · Cited by 2,276 · see citing works
Using AUC and accuracy in evaluating learning algorithms
Huang, J., & Ling, C. X. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.50 · Cited by 2,148 · see citing works
Assessing the accuracy of prediction algorithms for classification: an overview
Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A. F., & Nielsen, H. (2000) · Bioinformatics · doi:10.1093/bioinformatics/16.5.412 · Cited by 2,111 · see citing works
IEEE Transactions on Knowledge and Data Engineering
(2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.69 · Cited by 1,671 · see citing works
García, S., & Herrera, F. (2008) · Cited by 1,277 · see citing works
Information Theory and Coding.
Good, I. J., & Abramson, N. (1964) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2343598 · Cited by 1,086 · see citing works
Measuring classifier performance: a coherent alternative to the area under the ROC curve
Hand, D. J. (2009) · Machine Learning · doi:10.1007/s10994-009-5119-5 · Cited by 1,037 · see citing works
MAQC Consortium (2010) · Nature Biotechnology · doi:10.1038/nbt.1665 · Cited by 889 · see citing works
Self-citationAn experimental comparison of performance measures for classification
Ferri, C., Hernández-Orallo, J., & Modroiu, R. (2009) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.08.010 · Cited by 889 · see citing works
The Elements of Statistical Learning
Sapatinas, T. (2003) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2004.298_11.x · Cited by 635 · see citing works
Comparing two K-category assignments by a K-category correlation coefficient
Gorodkin, J. (2004) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2004.09.006 · Cited by 545 · see citing works
Proceedings of the International Joint Conference on Neural Networks (IJCNN) 2005
(2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1555790 · Cited by 382 · see citing works
Information Processing and Management
大貝, 明. (1984) · Journal of Information Processing and Management · doi:10.1241/johokanri.27.603 · Cited by 275 · see citing works
Volume under the ROC Surface for Multi-class Problems
Ferri, C., Hernández‐Orallo, J., & Salido, M. Á. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-39857-8_12 · Cited by 163 · see citing works
Evaluating diagnostic tests: The area under the ROC curve and the balance of errors
Hand, D. J. (2010) · Statistics in Medicine · doi:10.1002/sim.3859 · Cited by 140 · see citing works
Multi-class ROC analysis from a multi-objective optimisation perspective
Everson, R., & Fieldsend, J. E. (2005) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.016 · Cited by 122 · see citing works
Efficient Multiclass ROC Approximation by Decomposition via Confusion Matrix Perturbation Analysis
Landgrebe, T. C. W., & Duin, R. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70740 · Cited by 120 · see citing works
Proceedings of the 12th European Conference on Machine Learning
Raedt, L. D., & Flach, P. A. (2001) · Cited by 111 · see citing works
A novel measure for evaluating classifiers
Wei, J., Yuan, X., Hu, Q., & Wang, S. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.11.040 · Cited by 65 · see citing works
Information Theoretic Feature Crediting in Multiclass Support Vector Machines
Sindhwani, V., Bhattacharya, P., & Rakshit, S. (2001) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972719.16 · Cited by 64 · see citing works
Visualization and analysis of classifiers performance in multi-class medical data
Di̇ri̇, B., & Albayrak, S. (2006) · Expert Systems with Applications · doi:10.1016/j.eswa.2006.10.016 · Cited by 54 · see citing works
Shi, L. M., Campbell, G. R., Jones, W. D., Campagne, F., Wen, Z., Walker, S. J., Su, Z., Chu, T., Goodsaid, F. M., Pusztai, L., et al. (2010) · Cited by 37 · see citing works
Self-citationMulti-class AUC metrics and weighted alternatives
CALSTER, B. V., Belle, V. V., Condous, G., Bourne, T., Timmerman, D., & Huffel, S. V. (2008) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2008.4633979 · Cited by 24 · see citing works
Confusion Matrix Disagreement for Multiple Classifiers
Freitas, C. O. D. A., de Carvalho, J. M., Oliveira, J., Aires, S. B. K., & Sabourin, R. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-76725-1_41 · Cited by 20 · see citing works
A fast and efficient gene-network reconstruction method from multiple over-expression experiments
Stokić, D., Hanel, R., & Thurner, S. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-253 · Cited by 17 · see citing works
A method to quantify the error distribution in confusion matrices
van Son, R. J. J. H. (1995) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1995-518 · Cited by 13 · see citing works
Reconstructing Linear Gene Regulatory Networks
Supper, J., Spieth, C., & Zell, A. (2007) · Springer eBooks · doi:10.1007/978-3-540-71783-6_26 · Cited by 9 · see citing works
Comparing Classification Results between N-ary and Binary Problems
Felkin, M. (2007) · Studies in computational intelligence · doi:10.1007/978-3-540-44918-8_12 · Cited by 8 · see citing works
Li, Y. (2022) · Digital Archive @ GSU · doi:10.57709/1059733 · Cited by 5 · see citing works
Biochimica et biophysica acta. Protein structure
ScienceDirect (1981) · Cited by 5 · see citing works
Distance-based Disagreement Classifiers Combination
Freitas, C. O. D. A., de Carvalho, J. M., Oliveira, J. J., Aires, S. B. K., & Sabourin, R. (2007) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/ijcnn.2007.4371390 · Cited by 2 · see citing works
Flagged first, then by citations
2 of the 43 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- [11]Landgrebe T, Duin R (2005) On Neyman-Pearson optimisation for multiclass classifiers. In: Proceedings 16th Annual Symposium of the Pattern Recognition Association of South Africa. PRASA, pp. 165–170.
- [12]Landgrebe T, Duin R (2006) A simplified extension of the Area under the ROC to the multiclass domain. In: Proceedings 17th Annual Symposium of the Pattern Recognition Association of South Africa. PRASA, pp. 241–245.
Cited by
Sorted by citations
Chicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143 · see citing works
Self-citationSendra, A. L., Carrasco, A., Martín-Gómez, A. M., & Heras, A. D. L. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.02.023 · Cited by 1,137 · see citing works
Chicco, D., Tötsch, N., & Jurman, G. (2021) · BioData Mining · doi:10.1186/s13040-021-00244-z · Cited by 893 · see citing works
Self-citationSiamese Neural Networks: An Overview
Chicco, D. (2020) · Methods in molecular biology · doi:10.1007/978-1-0716-0826-5_3 · Cited by 681 · see citing works
Chicco, D., & Jurman, G. (2023) · BioData Mining · doi:10.1186/s13040-023-00322-4 · Cited by 667 · see citing works
Self-citationChicco, D., & Jurman, G. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-1023-5 · Cited by 630 · see citing works
Self-citationNaser, M. Z., & Alavi, A. H. (2021) · Architecture Structures and Construction · doi:10.1007/s44150-021-00015-8 · Cited by 473 · see citing works
Boosting methods for multi-class imbalanced data classification: an experimental review
Tanha, J., Abdi, Y., Samadi, N., Razzaghi-Asl, N., & Asadpour, M. (2020) · Journal Of Big Data · doi:10.1186/s40537-020-00349-y · Cited by 457 · see citing works
Chicco, D., Warrens, M., & Jurman, G. (2021) · IEEE Access · doi:10.1109/access.2021.3084050 · Cited by 365 · see citing works
Self-citationAl–antari, M. A., Al‐masni, M. A., Choi, M., Han, S., & Kim, T. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.06.003 · Cited by 407 · see citing works
Jiang, Y., Chen, L., Zhang, H., & Xiao, X. (2019) · PLoS ONE · doi:10.1371/journal.pone.0214587 · Cited by 373 · see citing works
Multivariate comparison of classification performance measures
Ballabio, D., Grisoni, F., & Todeschini, R. (2017) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2017.12.004 · Cited by 331 · see citing works
Why Cohen’s Kappa should be avoided as performance measure in classification
Delgado, R., & Tibau, X.-A. (2019) · PLOS ONE · doi:10.1371/journal.pone.0222916 · Cited by 324 · see citing works
DNA methylation-based predictors of health: applications and statistical considerations
Yousefi, P., Suderman, M. J., Langdon, R., Whitehurst, O., Smith, G. D., & Relton, C. L. (2022) · Nature Reviews Genetics · doi:10.1038/s41576-022-00465-w · Cited by 228 · see citing works
Valverde-Albacete, F. J., & Peláez-Moreno, C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0084217 · Cited by 212 · see citing works
A mountable toilet system for personalized health monitoring via the analysis of excreta
Park, S.-M., Won, D. D., Lee, B. J., Escobedo, D., Esteva, A., Aalipour, A., Ge, T. J., Kim, J. H., Suh, S., Choi, E. H., et al. (2020) · Nature Biomedical Engineering · doi:10.1038/s41551-020-0534-9 · Cited by 175 · see citing works
Koçak, B., Yardımcı, A. H., Bektaş, C. T., Türkcanoğlu, M. H., Erdim, Ç., Yücetaş, U., Koca, S. B., & Kılıçkesmez, Ö. (2018) · European Journal of Radiology · doi:10.1016/j.ejrad.2018.08.014 · Cited by 135 · see citing works
Classification of Heart Sounds Using Convolutional Neural Network
Li, F., Tang, H. X., Shang, S., Mathiak, K., & Cong, F. (2020) · Applied Sciences · doi:10.3390/app10113956 · Cited by 129 · see citing works
Liu, F., Xu, K., Ding, W., Qiao, Y., & Wang, L. (2021) · Cement and Concrete Composites · doi:10.1016/j.cemconcomp.2021.104196 · Cited by 117 · see citing works
Chicco, D., Старовойтов, В. В., & Jurman, G. (2021) · IEEE Access · doi:10.1109/access.2021.3068614 · Cited by 112 · see citing works
Self-citationDeconstructing Cross-Entropy for Probabilistic Binary Classifiers
Ramos, D., Franco-Pedroso, J., Lozano-Diez, A., & González-Rodríguez, J. (2018) · Entropy · doi:10.3390/e20030208 · Cited by 111 · see citing works
Real-world application of machine-learning-based fault detection trained with experimental data
Bode, G., Thul, S., Baranski, M., & Müller, D. (2020) · Energy · doi:10.1016/j.energy.2020.117323 · Cited by 99 · see citing works
Prediction of N-linked glycosylation sites using position relative features and statistical moments
Akmal, M. A., Rasool, N., & Khan, Y. D. (2017) · PLoS ONE · doi:10.1371/journal.pone.0181966 · Cited by 98 · see citing works
Evaluation of single-cell classifiers for single-cell RNA sequencing data sets
Zhao, X. S., Wu, S., Fang, N., Sun, X. S., & Fan, J. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz096 · Cited by 97 · see citing works
Semantic Frames to Predict Stock Price Movement
Xie, B., Passonneau, R. J., Wu, L., & Creamer, G. G. (2013) · Cited by 100 · see citing works
Thriumani, R., Zakaria, A., Hashim, Y. Z. H.-Y., Jeffree, A. I., Helmy, K., Kamarudin, L. M., Omar, M. I., Shakaff, A. Y. M., Adom, A. H., & Persaud, K. C. (2018) · BMC Cancer · doi:10.1186/s12885-018-4235-7 · Cited by 89 · see citing works
Gene expression profiling gut microbiota in different races of humans
Chen, L., Zhang, Y., Huang, T., & Cai, Y. (2016) · Scientific Reports · doi:10.1038/srep23075 · Cited by 89 · see citing works
Detecting Drinking Episodes in Young Adults Using Smartphone-based Sensors
Bae, S. W., Ferreira, D., Suffoletto, B. P., Puyana, J. C., Kurtz, R., Chung, T. A., & Dey, A. K. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3090051 · Cited by 84 · see citing works
Classifiers and their Metrics Quantified
Brown, J. B. (2018) · Molecular Informatics · doi:10.1002/minf.201700127 · Cited by 82 · see citing works
Making Disk Failure Predictions SMARTer
Lu, S., Luo, B., Patel, T., Yao, Y., Tiwari, D., & Shi, W. (2020) · File and Storage Technologies · Cited by 81 · see citing works
In vivo Raman spectroscopy for detection of oral neoplasia: A pilot clinical study
Krishna, H., Majumder, S. K., Chaturvedi, P., Muttagi, S., & Gupta, P. K. (2013) · Journal of Biophotonics · doi:10.1002/jbio.201300030 · Cited by 81 · see citing works
Shahid, S. M., Ko, S.-H., & Kwon, S. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116233 · Cited by 77 · see citing works
Haque, M. N., Noman, N., Berretta, R. E., & Moscato, P. (2016) · PLoS ONE · doi:10.1371/journal.pone.0146116 · Cited by 77 · see citing works
Lopez‐Rincon, A., Mendoza‐Maldonado, L., Martínez‐Archundia, M., Schönhuth, A., Kraneveld, A. D., Garssen, J., & Tonda, A. P. (2020) · Cancers · doi:10.3390/cancers12071785 · Cited by 76 · see citing works
Identifying maximum imbalance in datasets for fault diagnosis of gearboxes
dos Santos, P. T. C., Maudes, J., & Bustillo, A. (2015) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-015-1110-0 · Cited by 75 · see citing works
Integrative Network Fusion: A Multi-Omics Approach in Molecular Profiling
Chierici, M., Bussola, N., Marcolini, A., Francescatto, M., Zandonà, A., Trastulla, L., Agostinelli, C., Jurman, G., & Furlanello, C. (2020) · Frontiers in Oncology · doi:10.3389/fonc.2020.01065 · Cited by 75 · see citing works
Self-citationProsodic analysis of neutral, stress-modified and rhymed speech in patients with Parkinson's disease
Galáž, Z., Mekyska, J., Mzourek, Z., Smékal, Z., Rektorová, I., Eliasova, I., Košťálová, M., Mračková, M., & Beránková, D. (2016) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2015.12.011 · Cited by 72 · see citing works
HiTSelect: a comprehensive tool for high-complexity-pooled screen analysis
Diaz, A. A., Qin, H., Ramalho‐Santos, M., & Song, J. S. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku1197 · Cited by 65 · see citing works
Speck‐Planche, A., Kleandrova, V. V., Ruso, J. M., & Cordeiro, M. N. D. S. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00630 · Cited by 64 · see citing works
Predicting Delivery Capability in Iterative Software Development
Choetkiertikul, M., Dam, H. K., Tran, T., Ghose, A., & Grundy, J. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2693989 · Cited by 64 · see citing works
StutterNet: Stuttering Detection Using Time Delay Neural Network
Sheikh, S. A., Sahidullah, M., Hirsch, F., & Ouni, S. (2021) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco54536.2021.9616063 · Cited by 60 · see citing works
Classifying Ten Types of Major Cancers Based on Reverse Phase Protein Array Profiles
Zhang, P., Chen, L., Huang, T., Zhang, N., Kong, X., & Cai, Y. (2015) · PLoS ONE · doi:10.1371/journal.pone.0123147 · Cited by 60 · see citing works
Evaluating reproducibility of AI algorithms in digital pathology with DAPPER
Bizzego, A., Bussola, N., Chierici, M., Maggio, V., Francescatto, M., Cima, L., Cristoforetti, M., Jurman, G., & Furlanello, C. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006269 · Cited by 58 · see citing works
Self-citationZhang, X., Akber, M. Z., & Zheng, W. (2022) · Journal of Building Engineering · doi:10.1016/j.jobe.2022.104997 · Cited by 59 · see citing works
Chicco, D., & Jurman, G. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104426 · Cited by 55 · see citing works
Self-citationXiong, X., Wang, J., Hu, S., Dai, Y., Zhang, Y., & Hu, C. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2021.601699 · Cited by 52 · see citing works
Integrating deep and radiomics features in cancer bioimaging
Bizzego, A., Bussola, N., Salvalai, D., Chierici, M., Maggio, V., Jurman, G., & Furlanello, C. (2019) · IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) · doi:10.1109/cibcb.2019.8791473 · Cited by 50 · see citing works
Self-citationGenerating and evaluating synthetic data in digital pathology through diffusion models
Pozzi, M., Noei, S., Robbi, E., Cima, L., Moroni, M., Munari, E., Torresani, E., & Jurman, G. (2024) · Scientific Reports · doi:10.1038/s41598-024-79602-w · Cited by 51 · see citing works
Self-citationAlexandridis, A., & Chondrodima, E. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.03.008 · Cited by 47 · see citing works
Chatterjee, S., Sarkar, S., Dey, N., Sen, S., Goto, T., & Debnath, N. C. (2017) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin.2017.8104902 · Cited by 47 · see citing works
Enhancing energy efficiency in the residential sector with smart meter data analytics
Hopf, K., Sodenkamp, M., & Staake, T. (2018) · Electronic Markets · doi:10.1007/s12525-018-0290-9 · Cited by 47 · see citing works
Pear Defect Detection Method Based on ResNet and DCGAN
Zhang, Y., Wa, S., Sun, P., & Wang, Y. (2021) · Information · doi:10.3390/info12100397 · Cited by 46 · see citing works
Xu, K., Jin, Q., Li, J., Ushizima, D. M., Li, V. C., Kurtis, K. E., & Monteiro, P. J. M. (2023) · Cement and Concrete Research · doi:10.1016/j.cemconres.2023.107164 · Cited by 45 · see citing works
The MCC-F1 curve: a performance evaluation technique for binary classificationPreprint
Cao, C., Chicco, D., & Hoffman, M. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11278 · Cited by 45 · see citing works
Chicco, D., & Jurman, G. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.876814 · Cited by 44 · see citing works
Self-citationComparison of High-Level Microarray Analysis Methods in the Context of Result Consistency
Chromiński, K., & Tkacz, M. A. (2015) · PLoS ONE · doi:10.1371/journal.pone.0128845 · Cited by 44 · see citing works
Allesøe, R. L., Thompson, W. K., Bybjerg‐Grauholm, J., HOUGAARD, D. M., Nordentoft, M., Werge, T. M., Rasmussen, S., & Benros, M. E. (2022) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2022.4076 · Cited by 43 · see citing works
Pearson–Matthews correlation coefficients for binary and multinary classification
Stoica, P., & Babu, P. (2024) · Signal Processing · doi:10.1016/j.sigpro.2024.109511 · Cited by 43 · see citing works
Cheah, B. C. J., Vicente, C. R., & Chan, K. R. (2025) · Viruses · doi:10.3390/v17070882 · Cited by 43 · see citing works
Detection of wheezes using their signature in the spectrogram space and musical features
Mendes, L. G. A., Vogiatzis, I., Perantoni, E., Kaimakamis, E., Chouvarda, I., Maglaveras, N., Tsara, V., Teixeira, C., Carvalho, P., Henriques, J. M. O., et al. (2015) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7319657 · Cited by 43 · see citing works
Speck‐Planche, A., & Cordeiro, M. N. D. S. (2014) · Future Medicinal Chemistry · doi:10.4155/fmc.14.136 · Cited by 42 · see citing works
Salari, N., Shohaimi, S., Najafi, F., Nallappan, M., & Karishnarajah, I. (2014) · PLoS ONE · doi:10.1371/journal.pone.0112987 · Cited by 42 · see citing works
Wang, Y., Bi, Z., Xie, Y., Wu, T., Zeng, X., Chen, S., & Zhou, D. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3102090 · Cited by 42 · see citing works
Learning from adversarial medical images for X-ray breast mass segmentation
Shen, T., Gou, C., Wang, F., He, Z., & Chen, W. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.105012 · Cited by 41 · see citing works
Andreu-Pérez, J., Leff, D. R., Shetty, K., Darzi, A. W., & Yang, G. (2016) · Brain Connectivity · doi:10.1089/brain.2015.0350 · Cited by 40 · see citing works
Brown, P., Tan, A.-C., El-Esawi, M. A., Liehr, T., Blanck, O., Gladue, D. P., Almeida, G. M. D. F., Cernava, T., Sorzano, C. Ó. S., Yeung, A. W. K., et al. (2019) · Database · doi:10.1093/database/baz085 · Cited by 42 · see citing works
Bulavas, V., Marcinkevičius, V., & Rumiński, J. (2021) · Informatica · doi:10.15388/21-infor457 · Cited by 44 · see citing works
Joshi, R. C., Usmani, K., Krishnan, G. S., Blackmon, F. A., & Javidi, B. (2023) · Optics Express · doi:10.1364/oe.510681 · Cited by 39 · see citing works
Raza, M. A., Hussain, F. K., Hussain, F. K., Zhao, M. H., & Rehman, Z. U. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.06.022 · Cited by 39 · see citing works
Liang, Z., & Martell, M. A. C. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.665946 · Cited by 40 · see citing works
Privacy-Preserving Household Characteristic Identification With Federated Learning Method
Lin, J., Ma, J., & Zhu, J. G. (2021) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2021.3125677 · Cited by 39 · see citing works
Daily Locomotion Recognition and Prediction: A Kinematic Data-Based Machine Learning Approach
Figueiredo, J., Carvalho, S. P., Goncalve, D., Moreno, J. C., & Santos, C. P. (2020) · IEEE Access · doi:10.1109/access.2020.2971552 · Cited by 37 · see citing works
Predicting building age from urban form at large scale
Nachtigall, F., Milojevic-Dupont, N., Wagner, F., & Creutzig, F. (2023) · Computers Environment and Urban Systems · doi:10.1016/j.compenvurbsys.2023.102010 · Cited by 37 · see citing works
Kellner, L., Stender, M., Polach, F. V. B. U., Herrnring, H., Ehlers, S., Hoffmann, N., & Høyland, K. V. (2019) · Cold Regions Science and Technology · doi:10.1016/j.coldregions.2019.02.007 · Cited by 37 · see citing works
Young driver fatal motorcycle accident analysis by jointly maximizing accuracy and information
Halbersberg, D., & Lerner, B. (2019) · Accident Analysis & Prevention · doi:10.1016/j.aap.2019.04.016 · Cited by 37 · see citing works
A Pan-Cancer Approach to Predict Responsiveness to Immune Checkpoint Inhibitors by Machine Learning
Polano, M., Chierici, M., Bo, M. D., Gentilini, D., Cintio, F. D., Baboçi, L., Gibbs, D. L., Furlanello, C., & Toffoli, G. (2019) · Cancers · doi:10.3390/cancers11101562 · Cited by 37 · see citing works
Self-citationValuing free-form text data from maintenance logs through transfer learning with CamemBERT
Cadavid, J. P. U., Grabot, B., Lamouri, S., Pellerin, R., & Fortin, A. (2020) · Enterprise Information Systems · doi:10.1080/17517575.2020.1790043 · Cited by 38 · see citing works
Deep Neural Networks and Transfer Learning on a Multivariate Physiological Signal Dataset
Bizzego, A., Gabrieli, G., & Esposito, G. (2021) · Bioengineering · doi:10.3390/bioengineering8030035 · Cited by 36 · see citing works
Nascimento, M. H. C., Marcarini, W. D., Folli, G. S., Filho, W. G. D. S., Barbosa, L. L., de Paulo, E. H., Vassallo, P. F., Mill, J. G., Baraúna, V. G., Martin, F. L., et al. (2022) · Analytical Chemistry · doi:10.1021/acs.analchem.1c04162 · Cited by 37 · see citing works
A computer-aided diagnosis system for breast ultrasound based on weighted BI-RADS classes
Rodríguez-Cristerna, A., Gómez‐Flores, W., & Pereira, W. C. D. A. (2017) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2017.10.004 · Cited by 36 · see citing works
Host Taxon Predictor - A Tool for Predicting Taxon of the Host of a Newly Discovered Virus
Gałan, W., Bąk, M., & Jakubowska, M. (2019) · Scientific Reports · doi:10.1038/s41598-019-39847-2 · Cited by 35 · see citing works
Davari, A., Islam, S., Seehaus, T., Hartmann, A., Braun, M., Maier, A., & Christlein, V. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3115883 · Cited by 35 · see citing works
Hamerla, G., Meyer, H., Hambsch, P. K., Wolf, U., Kuhnt, T., Hoffmann, K., & Surov, A. (2019) · Cancers · doi:10.3390/cancers11111680 · Cited by 34 · see citing works
Prediction of kinase inhibitors binding modes with machine learning and reduced descriptor sets
Abdelbaky, I., Tayara, H., & Chong, K. T. (2021) · Scientific Reports · doi:10.1038/s41598-020-80758-4 · Cited by 35 · see citing works
A molecular barcode and web-based data analysis tool to identify imported Plasmodium vivax malaria
Trimarsanto, H., Amato, R., Pearson, R. D., Sutanto, E., Noviyanti, R., Trianty, L., Marfurt, J., Pava, Z., Echeverry, D. F., Lopera-Mesa, T. M., et al. (2022) · Communications Biology · doi:10.1038/s42003-022-04352-2 · Cited by 34 · see citing works
Comparative study of quality estimation of binary classification
Старовойтов, В. В., & Golub, Y. I. (2020) · Informatics · doi:10.37661/1816-0301-2020-17-1-87-101 · Cited by 33 · see citing works
Prediction of fatigue failure in small-scale butt-welded joints with explainable machine learning
Braun, M., Kellner, L., Schreiber, S., & Ehlers, S. (2022) · Procedia Structural Integrity · doi:10.1016/j.prostr.2022.03.019 · Cited by 33 · see citing works
Wang, L., Mo, T., Wang, X., Chen, W., He, Q., Li, X., Zhang, S., Yang, R., Wu, J., Gu, X., et al. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106517 · Cited by 32 · see citing works
Kumar, D., Verma, C., Dahiya, S., Singh, P. K., Raboacă, M. S., Illés, Z., & Bakariya, B. (2021) · Sensors · doi:10.3390/s21196584 · Cited by 32 · see citing works
Instance space analysis and algorithm selection for the job shop scheduling problem
Strassl, S., & Musliu, N. (2022) · Computers & Operations Research · doi:10.1016/j.cor.2021.105661 · Cited by 32 · see citing works
AbuAlkebash, H., Saleh, R. A. A., & Ertunç, H. M. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107934 · Cited by 31 · see citing works
Advanced Parametrization of Graphomotor Difficulties in School-Aged Children
Galáž, Z., Mucha, J., Zvončák, V., Mekyska, J., Smékal, Z., Šafárová, K., Ondrackova, A., Urbánek, T., Havigerová, J. M., Bednářová, J., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3003214 · Cited by 33 · see citing works
The flood cooking book: ingredients and regional flavors of floods across Germany
Merz, R., Tarasova, L., & Basso, S. (2020) · Environmental Research Letters · doi:10.1088/1748-9326/abb9dd · Cited by 30 · see citing works
Exploration of Convolutional Neural Network models for source code classification
Barchi, F., Parisi, E., Urgese, G., Ficarra, E., & Acquaviva, A. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.104075 · Cited by 30 · see citing works
Bosch, D. D., Ledo, J., & Queralt, P. (2013) · Surveys in Geophysics · doi:10.1007/s10712-013-9242-2 · Cited by 30 · see citing works
Xie, X., & Li, A. (2018) · Geoderma · doi:10.1016/j.geoderma.2018.03.029 · Cited by 30 · see citing works
Robust Deep-Learning-Based Road-Prediction for Augmented Reality Navigation Systems at Night
Limmer, M., Forster, J., Baudach, D., Schüle, F., Schweiger, R., & Lensch, H. P. A. (2016) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2016.7795862 · Cited by 30 · see citing works
Delgado-Font, W., Escobedo-Nicot, M., González-Hidalgo, M., Herold-García, S., Jaume-i-Capó, A., & Mir, A. (2020) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-019-02085-9 · Cited by 29 · see citing works
Interpretable Machine Learning Assessment
Han, H., Wu, Y., Wang, J., & Han, A. (2022) · Neurocomputing · doi:10.1016/j.neucom.2023.126891 · Cited by 29 · see citing works
Diagnosis system for cancer disease using a single setting approach
Bhuyan, H. K., Vijayaraj, A., & Ravi, V. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15478-8 · Cited by 28 · see citing works
- Authors
- Jurman, G., Riccadonna, S., & Furlanello, C.
- Affiliations
- Fondazione Bruno Kessler (IT)
- Corresponding author
- Jurman, Giuseppe
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-08-08
- Volume
- 7
- Issue
- 8
- Pages
- e41882
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3414515
- OpenAlex
- W1988301066
- Cited by
- 406
- References
- 43
- Citations by year
- 2026: 17 · 2025: 29 · 2024: 38 · 2023: 54 · 2022: 47 · 2021: 54 · 2020: 32 · 2019: 36 · 2018: 25 · 2017: 23 · 2016: 15 · 2015: 12 · 2014: 14 · 2013: 10
- FWCI
- 11.34
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- European Commission
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
- Keywords
- Matthews correlation coefficient, multi-class classification, generalization, binary classification, monotonicity
- MeSH terms
- Algorithms, Artificial Intelligence, Computers, Genotype, Probability, ROC Curve, Software, Statistics as Topic, Reproducibility of Results, Models, Statistical, Entropy, Computational Biology, Area Under Curve, Oligonucleotide Array Sequence Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
