100% Classification Accuracy Considered Harmful: The Normalized Information Transfer Factor Explains the Accuracy Paradox
Open accessAbstract
The most widely spread measure of performance, accuracy, suffers from a paradox: predictive models with a given level of accuracy may have greater predictive power than models with higher accuracy. Despite optimizing classification error rate, high accuracy models may fail to capture crucial information transfer in the classification task. We present evidence of this behavior by means of a combinatorial analysis where every possible contingency matrix of 2, 3 and 4 classes classifiers are depicted on the entropy triangle, a more reliable information-theoretic tool for classification assessment. Motivated by this, we develop from first principles a measure of classification performance that takes into consideration the information learned by classifiers. We are then able to obtain the entropy-modulated accuracy (EMA), a pessimistic estimate of the expected accuracy with the influence of the input distribution factored out, and the normalized information transfer factor (NIT), a measure of how efficient is the transmission of information from the input to the output set of classes. The EMA is a more natural measure of classification performance than accuracy when the heuristic to maximize is the transfer of information through the classifier instead of classification error count. The NIT factor measures the effectiveness of the learning process in classifiers and also makes it harder for them to "cheat" using techniques like specialization, while also promoting the interpretability of results. Their use is demonstrated in a mind reading task competition that aims at decoding the identity of a video stimulus based on magnetoencephalography recordings. We show how the EMA and the NIT factor reject rankings based in accuracy, choosing more meaningful and interpretable classifiers.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Traditional ML classification metrics propagate underlying biases, ignore the trustworthiness of negative instances, and do not take into account chance-level performance.
“These traditional metrics, however, propagate the underlying biases, ignore the trustworthiness of negative instances and do not take into account the chance-level performance [79,81].”
From A Survey of Underwater Acoustic Data Classification Methods Using Deep Learning for Shoreline Surveillance (2022) · cites this paper as [81]
The Introduction (page unavailable) explains that accuracy can favor a majority-class classifier on imbalanced data, and Results > Two performance measures based on perplexity discusses the accuracy of a clueless classifier. These passages support concerns about class bias and chance performance, but the paper does not establish that traditional metrics generally ignore the trustworthiness of negative instances. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · see citing works
Measuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058 · see citing works
A systematic analysis of performance measures for classification tasks
Sokolova, M., & Lapalme, G. (2009) · Information Processing & Management · doi:10.1016/j.ipm.2009.03.002 · Cited by 7,515 · see citing works
The use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217 · see citing works
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440 · see citing works
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399 · cited 2× · see citing works
“A comparison of the classical Matthew Correlation Coefficient (MCC) [28] and the Confusion Entropy (CEN) [27]—whose similarities are also explored in [10]–on three different classifications tasks: the MEG Mind Reading task already explored, the TASS sentiment analysis task [29]–both machine learning tasks—and the well…”
MethodSpeech and language processing
Politt, R., Pollock, J., & Waller, E. (2010) · Taylor & Francis eBooks · doi:10.4324/9780203461891_chapter_3 · Cited by 3,594 · see citing works
Assessing the Accuracy of Remotely Sensed Data
Congalton, R. G., & Green, K. (1998) · Mapping sciences series/Mapping science · doi:10.1201/9781420048568 · Cited by 3,434 · see citing works
Statistical methods for speech recognition
Jelinek, F. (1997) · Journal of the American Statistical Association · doi:10.2307/2670189 · Cited by 2,376 · see citing works
An Analysis of Perceptual Confusions Among Some English Consonants
Miller, G. A., & Nicely, P. E. (1955) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907526 · Cited by 1,885 · see citing works
Comparing clusterings—an information based distance
Meilă, M. (2006) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2006.11.013 · Cited by 1,603 · see citing works
Predicting the clinical status of human breast cancer by using gene expression profiles
West, M., Blanchette, C., Dressman, H. K., Huang, E. S., Ishida, S., Spang, R., Zuzan, H., Olson, J. A., Marks, J. R., & Nevins, J. R. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.201162998 · Cited by 1,318 · see citing works
Transmission of Information: A Statistical Theory of Communications
Fano, R. M., & Hawkins, D. (1961) · American Journal of Physics · doi:10.1119/1.1937609 · Cited by 870 · see citing works
Knowledge Discovery and Data Mining
Zhu, X., & Davidson, I. A. K. (2007) · IGI Global eBooks · doi:10.4018/978-1-59904-252-7 · Cited by 604 · see citing works
A Comparison of MCC and CEN Error Measures in Multi-Class Prediction
Jurman, G., Riccadonna, S., & Furlanello, C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0041882 · Cited by 406 · see citing works
Classification: Purposes, Principles, Progress, Prospects
Sokal, R. R. (1974) · Science · doi:10.1126/science.185.4157.1115 · Cited by 395 · see citing works
Information-Based Evaluation Criterion for Classifier's Performance
Kononenko, I., & Ivan Bratko, A. (1991) · Machine Learning · doi:10.1023/a:1022642017308 · Cited by 208 · see citing works
A lot of randomness is hiding in accuracy
Ben‐David, A. (2007) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2007.01.001 · Cited by 108 · see citing works
Knowledge Discovery and Data Mining: Challenges and Realities
Xingquan Zhu, A., & Davidson, I. A. K. (2007) · Cited by 105 · see citing works
Fish recruitment prediction, using robust supervised classification methods
Fernandes, J. A., Irigoien, X., Goikoetxea, N., Lozano, J. A., Inza, I., Pérez, A., & Bode, A. (2010) · Ecological Modelling · doi:10.1016/j.ecolmodel.2009.09.020 · Cited by 80 · see citing works
A novel measure for evaluating classifiers
Wei, J., Yuan, X., Hu, Q., & Wang, S. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.11.040 · Cited by 65 · see citing works
Information Theoretic Feature Crediting in Multiclass Support Vector Machines
Sindhwani, V., Bhattacharya, P., & Rakshit, S. (2001) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972719.16 · Cited by 64 · see citing works
Huang, H., Liu, C.-C., & Zhou, X. J. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0912043107 · Cited by 51 · see citing works
Exploring the within- and between-class correlation distributions for tumor classification
Wei, X., & Li, K.-C. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0910140107 · Cited by 31 · see citing works
Improvement in minority attack detection with skewness in network traffic
Thomas, C., & Balakrishnan, N. (2008) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.785623 · Cited by 28 · see citing works
Valverde-Albacete, F. J., Carrillo‐de‐Albornoz, J., & Peláez-Moreno, C. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40802-1_5 · Cited by 27 · see citing works
Self-citationData Balancing for Efficient Training of Hybrid ANN/HMM Automatic Speech Recognition Systems
García-Moral, A. I., Solera-Ureña, R., Peláez-Moreno, C., & Díaz-de-María, F. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2050513 · Cited by 23 · see citing works
Self-citationTwo information-theoretic tools to assess the performance of multi-class classifiers
Valverde-Albacete, F. J., & Peláez-Moreno, C. (2010) · Pattern Recognition Letters · doi:10.1016/j.patrec.2010.05.017 · Cited by 19 · see citing works
Self-citationICANN/PASCAL2 Challenge: MEG Mind Reading — Overview and Results
Arto Klami, A., Ramkumar, P., Virtanen, S., Parkkonen, L. T., Hari, R., & Samuel Kaski, A. (2012) · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
PANNZER —A practical tool for protein function prediction
Törönen, P., & Holm, L. (2021) · Protein Science · doi:10.1002/pro.4193 · Cited by 197 · see citing works
Benchmarking datasets for Anomaly-based Network Intrusion Detection: KDD CUP 99 alternatives
Divekar, A., Parekh, M., Savla, V., Mishra, R., & Shirole, M. R. (2018) · International Conference on Computing, Communication and Security (ICCCS) · doi:10.1109/cccs.2018.8586840 · Cited by 177 · see citing works
Benedetto, U., Dimagli, A., Sinha, S., Cocomello, L., Gibbison, B., Caputo, M., Gaunt, T. R., Lyon, M., Holmes, C., & Angelini, G. D. (2020) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2020.07.105 · Cited by 142 · see citing works
Malek, N. H. A., Yaacob, W. F. W., Nasir, S. A. M., & Shaadan, N. (2022) · Water · doi:10.3390/w14071067 · Cited by 129 · see citing works
Canbek, G., Sağıroğlu, Ş., Temizel, T. T., & Baykal, N. (2017) · doi:10.1109/ubmk.2017.8093539 · Cited by 111 · see citing works
Zhang, Y., Zhang⋆, Y., Hou, X., Chen, H., & Wang, S. (2017) · Multimedia Tools and Applications · doi:10.1007/s11042-017-4554-8 · Cited by 108 · see citing works
Brewster, L. R., Dale, J. J., Guttridge, T. L., Gruber, S. H., Hansell, A. C., Elliott, M., Cowx, I. G., Whitney, N. M., & Gleiss, A. C. (2018) · Marine Biology · doi:10.1007/s00227-018-3318-y · Cited by 104 · see citing works
New fuzzy C-means clustering method based on feature-weight and cluster-weight learning
Hashemzadeh, M., Oskouei, A. G., & Farajzadeh, N. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.02.038 · Cited by 103 · see citing works
Domingos, L. C. F., Santos, P. E., Skelton, P. S. M., Brinkworth, R. S. A., & Sammut, K. (2022) · Sensors · doi:10.3390/s22062181 · Cited by 93 · see citing works
What’s Happening Around the World? A Survey and Framework on Event Detection Techniques on Twitter
Saeed, Z., Abbasi, R. A., Maqbool, O., Sadaf, A., Razzak, I., Daud, A. Y., Aljohani, N. R., & Xu, G. (2019) · Journal of Grid Computing · doi:10.1007/s10723-019-09482-2 · Cited by 86 · see citing works
Automatic segmentation of the pharyngeal airway space with convolutional neural network
Shujaat, S., Jazil, O., Willems, H., Gerven, A. V., Shaheen, E., Politis, C., & Jacobs, R. (2021) · Journal of Dentistry · doi:10.1016/j.jdent.2021.103705 · Cited by 83 · see citing works
Peeking into a black box, the fairness and generalizability of a MIMIC-III benchmarking model.
Röösli, E., Bozkurt, S., & Hernandez-Boussard, T. (2022) · Scientific Data · doi:10.1038/s41597-021-01110-7 · Cited by 73 · see citing works
Rahman, J. S., Gedeon, T., Caldwell, S., Jones, R., & Jin, Z. (2020) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2021-0001 · Cited by 78 · see citing works
Dang, T. K., Tran, T. C., Tuan, L. M., & Tiep, M. V. (2021) · Applied Sciences · doi:10.3390/app112110004 · Cited by 69 · see citing works
Wang, X., & Kim, S. H. (2019) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/0361198119844456 · Cited by 69 · see citing works
Canbek, G., Temizel, T. T., & Sağıroğlu, Ş. (2022) · SN Computer Science · doi:10.1007/s42979-022-01409-1 · Cited by 60 · see citing works
Modeling highly imbalanced crash severity data by ensemble methods and global sensitivity analysis
Jiang, L., Xie, Y., Wen, X. L., & Ren, T. (2020) · Journal of Transportation Safety & Security · doi:10.1080/19439962.2020.1796863 · Cited by 60 · see citing works
Rjoob, K. W. Y., Bond, R., Finlay, D., McGilligan, V., Leslie, S. J., Rababah, A., Iftikhar, A., Guldenring, D., Knoery, C., McShane, A., et al. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102381 · Cited by 51 · see citing works
BenchMetrics: a systematic benchmarking method for binary classification performance metrics
Canbek, G., Temizel, T. T., & Sağıroğlu, Ş. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06103-6 · Cited by 51 · see citing works
Pusil, S. A., Dimitriadis, S. I., López, M. E., Pereda, E., & Maestú, F. (2019) · NeuroImage Clinical · doi:10.1016/j.nicl.2019.101972 · Cited by 50 · see citing works
Lu, S.-C., Xu, C., Nguyen, C., Geng, Y., Pfob, A., & Sidey-Gibbons, C. J. (2022) · JMIR Medical Informatics · doi:10.2196/33182 · Cited by 49 · see citing works
Machine Learning for Prediction of Imbalanced Data: Credit Fraud Detection
Tran, T. C., & Dang, T. K. (2021) · International Conference on Ubiquitous Information Management and Communication (IMCOM) · doi:10.1109/imcom51814.2021.9377352 · Cited by 42 · see citing works
The Detection of Depression Using Multimodal Models Based on Text and Voice Quality Features
Solieman, H., & Pustozerov, E. (2021) · IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering · doi:10.1109/elconrus51938.2021.9396540 · Cited by 41 · see citing works
Bindi: Affective Internet of Things to Combat Gender-Based Violence
Miranda, J., Rituerto-González, E., Luis-Mingueza, C., Canabal, M. F., Barcenas, A. R., Lanza-Gutiérrez, J. M., Peláez-Moreno, C., & López-Ongil, C. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3177256 · Cited by 44 · see citing works
Self-citationEdo-Osagie, O., Smith, G., LAKE, I. R., Edeghere, O., & de la Iglesia, B. (2019) · PLoS ONE · doi:10.1371/journal.pone.0210689 · Cited by 37 · see citing works
Predicting Bitcoin Prices Using Machine Learning
Dimitriadou, A., & Andros Gregoriou, A. (2023) · Entropy · doi:10.3390/e25050777 · Cited by 37 · see citing works
Threat Image Projection (TIP) into X-ray images of cargo containers for training humans and machines
Rogers, T. W., Jaccard, N., Protonotarios, E. D., Ollier, J., Morton, E. J., & Griffin, L. D. (2016) · IEEE International Carnahan Conference on Security Technology (ICCST) · doi:10.1109/ccst.2016.7815717 · Cited by 37 · see citing works
Forecasting Hospital Readmissions with Machine Learning
Michailidis, P., Dimitriadou, A., Papadimitriou, T., & Gogas, P. (2022) · Healthcare · doi:10.3390/healthcare10060981 · Cited by 36 · see citing works
Kim, J. K., Han, Y. S., & Lee, J. S. (2017) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.4128 · Cited by 36 · see citing works
Swaity, A., Elgarba, B. M., Morgan, N., Ali, S., Shujaat, S., Borsci, E., Chilvarquer, I., & Jacobs, R. (2024) · Scientific Reports · doi:10.1038/s41598-023-49613-0 · Cited by 34 · see citing works
Comparative study of quality estimation of binary classification
Старовойтов, В. В., & Golub, Y. I. (2020) · Informatics · doi:10.37661/1816-0301-2020-17-1-87-101 · Cited by 33 · see citing works
Adamo, M., Tomaselli, V., Tarantino, C., Vicario, S., Veronico, G., Lucas, R., & Blonda, P. N. (2020) · Remote Sensing · doi:10.3390/rs12091447 · Cited by 33 · see citing works
Predicting gas migration through existing oil and gas wells
Montague, J. A., Pinder, G. F., & Watson, T. L. (2018) · Environmental Geosciences · doi:10.1306/eg.01241817008 · Cited by 30 · see citing works
Machine Learning Analysis Reveals Novel Neuroimaging and Clinical Signatures of Frailty in HIV
Paul, R. H., Cho, K. S., Luckett, P. H., Strain, J. F., Belden, A. C., Bolzenius, J. D., Navid, J., Garcia-Egan, P. M., Cooley, S., Wisch, J. K., et al. (2020) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/qai.0000000000002360 · Cited by 30 · see citing works
Machine learning-based prognostic model for 30-day mortality prediction in Sepsis-3
Rahman, M. S., Islam, K. R., Prithula, J., Kumar, J., Mahmud, M., Alam, M. F., Reaz, M. B. I., Alqahtani, A., & Chowdhury, M. E. H. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02655-4 · Cited by 29 · see citing works
Wang, M., Zhang, J., Jiao, S., Zhang, X., Zhu, N., & Chen, G. (2020) · Scientometrics · doi:10.1007/s11192-020-03677-1 · Cited by 32 · see citing works
Exploring the performance of LBP-capsule networks with K-Means routing on complex images
Mensah, P. K., Weyori, B. A., & Ayidzoe, M. A. (2020) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2020.10.006 · Cited by 29 · see citing works
Text plagiarism classification using syntax based linguistic features
Kanjirangat, V., & Gupta, D. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.07.006 · Cited by 29 · see citing works
Hong, Y., Welch, C. J., Piras, P., & Tang, H. (2024) · Analytical Chemistry · doi:10.1021/acs.analchem.3c04028 · Cited by 28 · see citing works
Zaka, A., Mutahar, D., Gorcilov, J., Gupta, A. K., Kovoor, J. G., Stretton, B., Mridha, N., Sivagangabalan, G., Thiagalingam, A., Chow, C. K., et al. (2024) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztae074 · Cited by 28 · see citing works
Zhu, X., Koç, A., Morawski, R., & Le‐Ngoc, T. (2021) · IEEE Access · doi:10.1109/access.2021.3121750 · Cited by 28 · see citing works
Valverde-Albacete, F. J., Carrillo‐de‐Albornoz, J., & Peláez-Moreno, C. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40802-1_5 · Cited by 27 · see citing works
Self-citationA voting-based ensemble feature network for semiconductor wafer defect classification
Misra, S., Kim, D., Kim, J., Shin, W., & Kim, C. (2022) · Scientific Reports · doi:10.1038/s41598-022-20630-9 · Cited by 26 · see citing works
Fear Recognition for Women Using a Reduced Set of Physiological Signals
Miranda, J., Canabal, M. F., Gutiérrez-Martín, L., Lanza-Gutiérrez, J. M., Portela-Garcia, M., & López-Ongil, C. (2021) · Sensors · doi:10.3390/s21051587 · Cited by 25 · see citing works
Methodological and conceptual challenges in rare and severe event forecast verification
Ebert, P. A., & Milne, P. (2022) · Natural hazards and earth system sciences · doi:10.5194/nhess-22-539-2022 · Cited by 25 · see citing works
Has machine learning over-promised in healthcare?
Ge, W., Lueck, C. J., Suominen, H., & Apthorp, D. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102524 · Cited by 22 · see citing works
Xu, C., Subbiah, I. M., Lu, S.-C., Pfob, A., & Sidey-Gibbons, C. J. (2022) · Quality of Life Research · doi:10.1007/s11136-022-03284-y · Cited by 21 · see citing works
A Study of Disease Diagnosis Using Machine Learning
Poudel, S. (2022) · doi:10.3390/iech2022-12311 · Cited by 21 · see citing works
Heart Failure Emergency Readmission Prediction Using Stacking Machine Learning Model
Rahman, M. S., Rahman, H., Prithula, J., Chowdhury, M. E. H., Ahmed, M. U., Kumar, J., Murugappan, M., & Khan, M. S. (2023) · Diagnostics · doi:10.3390/diagnostics13111948 · Cited by 21 · see citing works
Enhancing Confusion Entropy (CEN) for binary and multiclass classification
Delgado, R., & Núñez-González, J. D. (2019) · PLoS ONE · doi:10.1371/journal.pone.0210264 · Cited by 21 · see citing works
Chu, Y.-W., Chang, K.-P., Chen, C.-W., Liang, Y.-T., Soh, Z. T., & Hsieh, L. (2020) · Scientific Reports · doi:10.1038/s41598-020-58336-5 · Cited by 21 · see citing works
The open-set problem in acoustic scene classification
Battaglino, D., Lepauloux, L., & Evans, N. (2016) · International Workshop on Acoustic Signal Enhancement · doi:10.1109/iwaenc.2016.7602939 · Cited by 20 · see citing works
Kia, S. M., Vega-Pons, S., Weisz, N., & Passerini, A. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00619 · Cited by 20 · see citing works
Garefalakis, T., Michelaraki, E., Roussou, S., Katrakazas, C., Brijs, T., & Yannis, G. (2024) · European Transport Research Review · doi:10.1186/s12544-024-00691-9 · Cited by 18 · see citing works
Machine learning analysis to classify nanoparticles from noisy spICP-TOFMS data
Johnson, R. L. B., & Gundlach‐Graham, A. (2023) · Journal of Analytical Atomic Spectrometry · doi:10.1039/d3ja00081h · Cited by 18 · see citing works
Forget the Myth of the Air Gap: Machine Learning for Reliable Intrusion Detection in SCADA Systems
Perez, R. L., Adamsky, F., Soua, R., & Engel, T. (2019) · ICST Transactions on Security and Safety · doi:10.4108/eai.25-1-2019.159348 · Cited by 18 · see citing works
Training radiomics-based CNNs for clinical outcome prediction: Challenges, strategies and findings
Pang, S., Field, M., Dowling, J., Vinod, S. K., Holloway, L. C., & Sowmya, A. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102230 · Cited by 18 · see citing works
Brain Melody Informatics: Analysing Effects of Music on Brainwave Patterns
Rahman, J. S., Gedeon, T., Caldwell, S., & Jones, R. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207392 · Cited by 18 · see citing works
Hernández, A. D., Owen, R., Nielsen, D. S., & McConville, R. (2023) · Journal of Responsible Innovation · doi:10.1080/23299460.2023.2222514 · Cited by 16 · see citing works
Nyström landmark sampling and regularized Christoffel functions
Fanuel, M., Schreurs, J., & Suykens, J. A. K. (2022) · Machine Learning · doi:10.1007/s10994-022-06165-0 · Cited by 16 · see citing works
Plyusnin, I., Holm, L., & Törönen, P. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007419 · Cited by 16 · see citing works
Diversity Sampling is an Implicit Regularization for Kernel Methods
Fanuel, M., Schreurs, J., & Suykens, J. A. K. (2021) · SIAM Journal on Mathematics of Data Science · doi:10.1137/20m1320031 · Cited by 15 · see citing works
Modelling Bathing Water Quality Using Official Monitoring Data
Džal, D., Kosović, I. N., Mastelić, T., Ivanković, D., Puljak, T., & Jozić, S. (2021) · Water · doi:10.3390/w13213005 · Cited by 15 · see citing works
Decisions are not all equal—Introducing a utility metric based on case-wise raters’ perceptions
Campagner, A., Sternini, F., & Cabitza, F. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106930 · Cited by 15 · see citing works
Lapp, L., Bouamrane, M.-M., Kavanagh, K., Roper, M., Young, D., & Schraag, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-21642-9_48 · Cited by 15 · see citing works
Min-Max Backpropagation Neural Network to Forecast e-Journal Visitors
Wibawa, A. P., Izdihar, Z. N., Utama, A. B. P., Hernandez, L., & Haviluddin, H. (2021) · International Conference on Artificial Intelligence in Information and Communication (ICAIIC) · doi:10.1109/icaiic51459.2021.9415197 · Cited by 14 · see citing works
Preference-Driven Classification Measure
Kozák, J. T., Probierz, B., Kania, K., & Juszczuk, P. (2022) · Entropy · doi:10.3390/e24040531 · Cited by 20 · see citing works
Pitoglou, S., Koumpouros, Y., & Αναστασίου, Α. (2018) · International Conference on Machine Learning and Data Engineering (iCMLDE) · doi:10.1109/icmlde.2018.00021 · Cited by 14 · see citing works
Tu, Y.-K., Kuo, C., Fang, S.-L., Chen, H., Chi, M.-K., Yao, M.-H., & Kuo, B. (2022) · Agriculture · doi:10.3390/agriculture12020259 · Cited by 15 · see citing works
Shi, X., Qu, T., Pottelbergh, G. V., van den Akker, M., & Moor, B. D. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.730748 · Cited by 14 · see citing works
Predicting Air Traffic Flow Management hotspots due to weather using Convolutional Neural Networks
Martínez, I. S., García‐Heras, J., Jardines, A., Cervantes, A., & Soler, M. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108014 · Cited by 13 · see citing works
Using Gaussian Mixture Models in plant morphometrics
Tiburtini, M., Scrucca, L., & Peruzzi, L. (2025) · Perspectives in Plant Ecology Evolution and Systematics · doi:10.1016/j.ppees.2025.125902 · Cited by 13 · see citing works
Evans, L., Owda, M., Crockett, K. A., & Vilas, A. F. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.03.019 · Cited by 13 · see citing works
Carvalho, B. G., Vargas, R. E. V., Salgado, R. M., Munaro, C. J., & Varejão, F. M. (2021) · IEEE International Symposium on Industrial Electronics (ISIE) · doi:10.1109/isie45552.2021.9576310 · Cited by 13 · see citing works
Performance of Human Annotators in Object Detection and Segmentation of Remotely Sensed Data
Blushtein-Livnon, R., Svoray, T., & Dorman, M. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3555235 · Cited by 12 · see citing works
The evaluation of data sources using multivariate entropy tools
Valverde-Albacete, F. J., & Peláez-Moreno, C. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.02.010 · Cited by 12 · see citing works
Self-citationResampling Techniques for Materials Informatics: Limitations in Crystal Point Groups Classification
Alsaui, A. A., Alghofaili, Y. A., Alghadeer, M., & Alharbi, F. H. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c00666 · Cited by 12 · see citing works
Hybrid Model for on-line Detection of Gas Turbine Lean Blowout Events
Iannitelli, M., Allegorico, C., Garau, F., & Capanni, M. (2018) · PHM Society European Conference · doi:10.36001/phme.2018.v4i1.405 · Cited by 12 · see citing works
A Data-Driven Approach for Determining Weights in Global Similarity Functions
Jaiswal, A. D., & Bach, K. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29249-2_9 · Cited by 12 · see citing works
Majdik, Z. P., & Wynn, J. (2022) · Technical Communication Quarterly · doi:10.1080/10572252.2022.2077452 · Cited by 11 · see citing works
Mladenova, T., Valova, I., Evstatiev, B. I., Valov, N., Varlyakov, I. S., Марков, Ц., STOYCHEVA, S., Mondeshka, L., & Markov, N. (2024) · AgriEngineering · doi:10.3390/agriengineering6030128 · Cited by 11 · see citing works
Gupta, A. K., Mustafiz, C., Mutahar, D., Zaka, A., Parvez, R., Mridha, N., Stretton, B., Kovoor, J. G., Bacchi, S., Ramponi, F., et al. (2025) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2025.01.037 · Cited by 11 · see citing works
Benchmarking Analysis of the Accuracy of Classification Methods Related to Entropy
Orenes, Y., Rabasa, A., Rodriguez-Sala, J. J., & Sánchez-Soriano, J. (2021) · Entropy · doi:10.3390/e23070850 · Cited by 11 · see citing works
Fundamental Fresh State Properties of Self‐Consolidating Concrete: A Meta‐Analysis of Mix Designs
García-Taengua, E. (2018) · Advances in Civil Engineering · doi:10.1155/2018/5237230 · Cited by 11 · see citing works
Deep transfer learning for the classification of variable sources
Kim, D.-W., Yeo, D., Bailer‐Jones, C. A. L., & Lee, G. (2021) · Astronomy and Astrophysics · doi:10.1051/0004-6361/202140369 · Cited by 11 · see citing works
Davis, K. E., & Maiden, R. (2021) · Studies in Social Science Research · doi:10.22158/sssr.v2n2p1 · Cited by 10 · see citing works
Metamorphic Testing-based Adversarial Attack to Fool Deepfake Detectors
Lim, N. T., Kuan, M. Y., Pu, M., Lim, M. K., & Chong, C. Y. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956543 · Cited by 10 · see citing works
Distinctive Phonetic Features Modeling and Extraction Using Deep Neural Networks
Seddiq, Y., Alotaibi, Y. A., Selouani, S., & Meftah, A. H. (2019) · IEEE Access · doi:10.1109/access.2019.2924014 · Cited by 10 · see citing works
Penalized maximum likelihood estimation of logit-based early warning systems
Pigini, C. (2021) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.01.004 · Cited by 9 · see citing works
HumDLoc: Human Protein Subcellular Localization Prediction Using Deep Neural Network
Semwal, R., & Varadwaj, P. K. (2020) · Current Genomics · doi:10.2174/1389202921999200528160534 · Cited by 9 · see citing works
Optimizing credit card fraud detection: a deep learning approach to imbalanced datasets
Ndama, O., Bensassi, I., & En-Naimi, E. M. (2024) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v14i4.pp4802-4814 · Cited by 9 · see citing works
Exploiting cepstral coefficients and CNN for efficient musical instrument classification
Mahanta, S. K., Basisth, N. J., Halder, E., Khilji, A. F. U. R., & Pakray, P. (2023) · Evolving Systems · doi:10.1007/s12530-023-09540-x · Cited by 9 · see citing works
Sequence-to-Sequence Acoustic-to-Phonetic Conversion Using Spectrograms and Deep Learning
Qamhan, M. A., Alotaibi, Y. A., Seddiq, Y., Meftah, A. H., & Selouani, S. (2021) · IEEE Access · doi:10.1109/access.2021.3083972 · Cited by 9 · see citing works
Data-Driven Estimation of a Driving Safety Tolerance Zone Using Imbalanced Machine Learning
Garefalakis, T., Katrakazas, C., & Yannis, G. (2022) · Sensors · doi:10.3390/s22145309 · Cited by 8 · see citing works
Alqalyoobi, S., Little, B. B., Oldham, J. M., & Obi, O. N. (2023) · Respiratory Research · doi:10.1186/s12931-023-02407-4 · Cited by 8 · see citing works
Shin, J., Carley, L. R., Dobson, G. B., & Carley, K. M. (2023) · Winter Simulation Conference · doi:10.1109/wsc60868.2023.10408211 · Cited by 8 · see citing works
Valverde-Albacete, F. J., & Peláez-Moreno, C. (2018) · Entropy · doi:10.3390/e20070498 · Cited by 8 · see citing works
Self-citationA Framework for Supervised Classification Performance Analysis with Information-Theoretic Methods
Valverde-Albacete, F. J., & Peláez-Moreno, C. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2915643 · Cited by 8 · see citing works
Self-citationOptimizing activity data collection with gamification points using uncertainty based active learning
Mairittha, N., Mairittha, T., & Inoue, S. (2019) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3341162.3345585 · Cited by 8 · see citing works
Credit Scoring with Drift Adaptation Using Local Regions of Competence
Nikolaidis, D., & Doumpos, M. (2022) · Operations Research Forum · doi:10.1007/s43069-022-00177-1 · Cited by 7 · see citing works
- Authors
- Valverde-Albacete, F. J., & Peláez-Moreno, C.
- Affiliations
- Universidad Nacional de Educación a Distancia (ES)
- Universidad Carlos III de Madrid (ES)
- Corresponding author
- Valverde-Albacete, Francisco J.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-01-10
- Volume
- 9
- Issue
- 1
- Pages
- e84217
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3888391
- OpenAlex
- W2015363484
- Cited by
- 212
- References
- 29
- Citations by year
- 2026: 13 · 2025: 12 · 2024: 27 · 2023: 14 · 2022: 35 · 2021: 38 · 2020: 20 · 2019: 21 · 2018: 12 · 2017: 8 · 2016: 8 · 2015: 1 · 2014: 2 · 2013: 1
- FWCI
- 6.31
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (acceptedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- e-Archivo (Carlos III University of Madrid) (submittedVersion, cc-by)
- Figshare (submittedVersion, cc-by)
- Funders
- European Commission (288024)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
- Keywords
- classification performance measures, contingency tables, information transmission, classifier evaluation, interpretability, theory of mind, magnetoencephalography, classification error rate
- MeSH terms
- Algorithms, Humans, Models, Theoretical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
