The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets
Open accessThe DOI matches its Crossref record; 55 of the 56 references it cites were found in a registry or an index.
Abstract
Binary classifiers are routinely evaluated with performance measures such as sensitivity and specificity, and performance is frequently illustrated with Receiver Operating Characteristics (ROC) plots. Alternative measures such as positive predictive value (PPV) and the associated Precision/Recall (PRC) plots are used less frequently. Many bioinformatics studies develop and evaluate classifiers that are to be applied to strongly imbalanced datasets in which the number of negatives outweighs the number of positives significantly. While ROC plots are visually appealing and provide an overview of a classifier's performance across a wide range of specificities, one can ask whether ROC plots could be misleading when applied in imbalanced classification scenarios. We show here that the visual interpretability of ROC plots in the context of imbalanced datasets can be deceptive with respect to conclusions about the reliability of classification performance, owing to an intuitive but wrong interpretation of specificity. PRC plots, on the other hand, can provide the viewer with an accurate prediction of future classification performance due to the fact that they evaluate the fraction of true positives among positive predictions. Our findings have potential implications for the interpretation of a large number of studies that use ROC plots on imbalanced datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“which is a more informative measure than ROC analysis in imbalanced scenarios59.”
From K-STAMM: a knowledge-enhanced spatial - temporal attention model with multimodal fusion for pneumonia prediction (2026) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Because extreme prediction is by definition an imbalanced classification problem, we base our analysis on precision-recall plots which are well-suited for this case [42].”
From Learning skillful medium-range global weather forecasting (2023) · cites this paper as [42]
In the citing paper
“Furthermore, careful consideration should be given when selecting evaluation metrics, as the precision-recall curve is more informative for imbalanced datasets than the most commonly used receiver-operating characteristic curve (true positive rate vs. false positive rate) (Fig. 5C).115”
From Use of artificial intelligence for liver diseases: A survey from the EASL congress 2024 (2024) · cites this paper as [115]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Bartel (2004) · doi:10.1016/s0092-8674(04)00045-5 · Cited by 34,714
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 3×
“Here, we review the theoretical background of commonly used evaluation measures, specifically ROC [24, 25], PRC [26], concentrated ROC (CROC) [27], and Cost Curves (CC) [28].”
BackgroundMethodThe meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
Bioconductor: open software development for computational biology and bioinformatics
Gentleman, R. C., Carey, V. J., Bates, D. M., Bolstad, B., Dettling, M., Dudoit, S., Ellis, B., Gautier, L., Ge, Y., Gentry, J., et al. (2004) · Genome Biology · doi:10.1186/gb-2004-5-10-r80 · Cited by 12,172 · cited 1×
“Several integrated machine learning and bioinformatics platforms such as WEKA [34] and Bioconductor [42, 43] also have basic functions or libraries for making ROC and PRC plots.”
MethodA training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
R: A Language for Data Analysis and Graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,220
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Cited by
Items 201–300 of 1,023
Machine learning approach to gene essentiality prediction: a review
Aromolaran, O., Aromolaran, D., Isewon, I., & Oyelade, J. O. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab128 · Cited by 120
Oh, S., Ji, H., Kim, J., Park, E., & del Pobil, A. P. (2022) · Information Technology & Tourism · doi:10.1007/s40558-022-00222-z · Cited by 143
Artificial intelligence for decision support in acute stroke — current roles and potential
Bivard, A., Churilov, L. P., & Parsons, M. W. (2020) · Nature Reviews Neurology · doi:10.1038/s41582-020-0390-y · Cited by 127
Van Calster, B., Collins, G. S., Vickers, A. J., Wynants, L., Kerr, K. F., Barreñada, L., Varoquaux, G., Singh, K., Moons, K. G., Hernandez-Boussard, T., et al. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100916 · Cited by 112
Lee, B., Baek, J., Park, S., & Yoon, S. (2016) · Proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics · doi:10.1145/2975167.2975212 · Cited by 102
A random forest based computational model for predicting novel lncRNA-disease associations
Yao, D., Zhan, X., Zhan, X., Kwoh, C. K., Peng, L., & Wang, J. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-3458-1 · Cited by 110
Machine learning-based statistical analysis for early stage detection of cervical cancer
Ali, M. M., Ahmed, K., Bui, F. M., Paul, B. K., Ibrahim, S. M., Quinn, J. M. W., & Moni, M. A. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104985 · Cited by 110
Principles of ipsilateral and contralateral cortico-cortical connectivity in the mouse
Goulas, A., Uylings, H. B. M., & Hilgetag, C. C. (2016) · Brain Structure and Function · doi:10.1007/s00429-016-1277-y · Cited by 108
Draelos, R. L., Dov, D., Mazurowski, M. A., Lo, J. Y.-C., Henao, R., Rubin, G. D., & Carin, L. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101857 · Cited by 165
An Improved Multi-Modal based Machine Learning Approach for the Prognosis of Alzheimer’s disease
Khan, A., & Zubair, S. (2020) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2020.04.004 · Cited by 130
Xuan, P., Sun, C., Zhang, T., Ye, Y., Shen, T., & Dong, Y. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00459 · Cited by 100
Yanagihara, R. T., Lee, C. S., Ting, D. S. W., & Lee, A. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.2.11 · Cited by 121
A practical guide to machine-learning scoring for structure-based virtual screening
Tran‐Nguyen, V., Junaid, M., Simeon, S., & Ballester, P. J. (2023) · Nature Protocols · doi:10.1038/s41596-023-00885-w · Cited by 115
Predicting student dropouts with machine learning: An empirical study in Finnish higher education
Vaarma, M., & Li, H. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102474 · Cited by 175
Mao, N., Shi, Y., Lian, C., Wang, Z., Zhang, K., Xie, H., Zhang, H., Chen, Q., Cheng, G., Xu, C., et al. (2022) · European Radiology · doi:10.1007/s00330-021-08414-7 · Cited by 99
Schubach, M., Re, M., Robinson, P. N., & Valentini, G. (2017) · Scientific Reports · doi:10.1038/s41598-017-03011-5 · Cited by 97
Weighted kappa measures for ordinal multi-class classification performance
Yılmaz, A. E., & Demirhan, H. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110020 · Cited by 133
Performance Measures for Binary Classification
Berrar, D. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-809633-8.20351-8 · Cited by 133
Predicting the Risk of Inpatient Hypoglycemia With Machine Learning Using Electronic Health Records
Ruan, Y., Bellot, A., Moysova, Z., Tan, G. D., Lumb, A., Davies, J., van der Schaar, M., & Rea, R. (2020) · Diabetes Care · doi:10.2337/dc19-1743 · Cited by 95
Moualla, S., Khorzom, K. I., & Jafar, A. (2021) · Computational Intelligence and Neuroscience · doi:10.1155/2021/5557577 · Cited by 121
Yuan, Q., Xie, J., Xie, J., Zhao, H., & Yang, Y. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad117 · Cited by 95
Validation of deep-learning-based triage and acuity score using a large national dataset
Kwon, J., Lee, Y., Lee, Y., Lee, S., Park, H., & Park, J. (2018) · PLoS ONE · doi:10.1371/journal.pone.0205836 · Cited by 102
Improved Alzheimer’s Disease Detection by MRI Using Multimodal Machine Learning Algorithms
Battineni, G., Hossain, M. A., Chintalapudi, N., Traini, E., Dhulipalla, V. R., Mariappan, R., & Amenta, F. (2021) · Diagnostics · doi:10.3390/diagnostics11112103 · Cited by 113
Quantitative analysis of plant ER architecture and dynamics
Pain, C., Kriechbaumer, V., Kittelmann, M., Hawes, C. R., & Fricker, M. D. (2019) · Nature Communications · doi:10.1038/s41467-019-08893-9 · Cited by 102
Nurmaini, S., Partan, R. U., Caesarendra, W., Dewi, T., Rahmatullah, M. N., Darmawahyuni, A., Bhayyu, V., & Firdaus, F. (2019) · Applied Sciences · doi:10.3390/app9142921 · Cited by 105
Chen, C., Hou, J., Shi, X., Yang, H. Q., Birchler, J. A., & Cheng, J. (2021) · BMC Bioinformatics · doi:10.1186/s12859-020-03952-1 · Cited by 105
Preventing rather than punishing: An early warning model of malfeasance in public procurement
Gallego, J., Rivero, G., & Martínez, J. D. S. (2020) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2020.06.006 · Cited by 140
Chiew, C. J., Liu, N., Tagami, T., Wong, T. H., Koh, Z. X., & Ong, M. E. H. (2019) · Medicine · doi:10.1097/md.0000000000014197 · Cited by 89
Chávez-Badiola, A., Farias, A. F.-S., Mendizabal‐Ruiz, G., Garcia-Sánchez, R., Drakeley, A., & García-Sandoval, J. P. (2020) · Scientific Reports · doi:10.1038/s41598-020-61357-9 · Cited by 106
Multireader Diagnostic Accuracy Imaging Studies: Fundamentals of Design and Analysis
Obuchowski, N. A., & Bullen, J. A. (2022) · Radiology · doi:10.1148/radiol.211593 · Cited by 109
A systematic review on automated clinical depression diagnosis
Mao, K., Wu, Y., & Chen, J. (2023) · npj Mental Health Research · doi:10.1038/s44184-023-00040-z · Cited by 127
Structure–activity relationship-based chemical classification of highly imbalanced Tox21 datasets
Idakwo, G., Thangapandian, S., Luttrell, J., Li, Y., Wang, N., Zhou, Z., Hong, H., Yang, B., Zhang, C., & Gong, P. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-020-00468-x · Cited by 99
Artificial intelligence in the fertility clinic: status, pitfalls and possibilities
Riegler, M. A., Stensen, M. H., Witczak, O., Andersen, J. M., Hicks, S. A., Hammer, H. L., Delbarre, E., Halvorsen, P., Yazidi, A., Holst, N., et al. (2021) · Human Reproduction · doi:10.1093/humrep/deab168 · Cited by 102
Mosquera, C., Ferrer, L., Milone, D. H., Luna, D. R., & Ferrante, E. (2024) · European Radiology · doi:10.1007/s00330-024-10834-0 · Cited by 127
Predicting polycystic ovary syndrome with machine learning algorithms from electronic health records
Zad, Z., Jiang, V. S., Wolf, A. T., Wang, T., Cheng, J. J., Paschalidis, I. C., & Mahalingaiah, S. (2024) · Frontiers in Endocrinology · doi:10.3389/fendo.2024.1298628 · Cited by 97
Alzheimer’s disease polygenic risk score as a predictor of conversion from mild-cognitive impairment
Chaudhury, S., Brookes, K. J., Patel, T., Fallows, A., Guetta-Baranes, T., Turton, J. C., Guerreiro, R., Bras, J., Hardy, J., Francis, P. T., et al. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0485-7 · Cited by 89
A Hybrid Model for Fast and Probabilistic Urban Pluvial Flood Prediction
Li, X., & Willems, P. (2020) · Water Resources Research · doi:10.1029/2019wr025128 · Cited by 101
Automated prediction of sepsis using temporal convolutional network
Kok, C., Jahmunah, V., Oh, S. L., Zhou, X., Gururajan, R., Tao, X., Cheong, K. H., Gururajan, R., Molinari, F., & Acharya, U. R. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103957 · Cited by 105
ST-Trans: Spatial-Temporal Transformer for Infrared Small Target Detection in Sequential Images
Tong, X., Zuo, Z., Su, S., Wei, J., Sun, X., Wu, P., & Zhao, Z. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3355947 · Cited by 97
Koçak, B., Klontzas, M. E., Stanzione, A., Meddeb, A., Demircioğlu, A., Blüthgen, C., Bressem, K. K., Ugga, L., Mercaldo, N. D., Díaz, O., et al. (2025) · European Journal of Radiology Artificial Intelligence · doi:10.1016/j.ejrai.2025.100030 · Cited by 148
Requirements Classification with Interpretable Machine Learning and Dependency Parsing
Dalpiaz, F., Dell’Anna, D., Aydemir, F. B., & Çevikol, S. (2019) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2019.00025 · Cited by 142
Exploring Impact of Age and Gender on Sentiment Analysis Using Machine Learning
Kumar, S., Gahalawat, M., Roy, P. P., Dogra, D. P., & Kim, B. (2020) · Electronics · doi:10.3390/electronics9020374 · Cited by 108
Integrated analysis of machine learning and deep learning in chili pest and disease identification
Loti, N. N. A., Noor, M. R. M., & Chang, S. (2020) · Journal of the Science of Food and Agriculture · doi:10.1002/jsfa.10987 · Cited by 108
Detecting white spot lesions on dental photography using deep learning: A pilot study
Askar, H., Krois, J., Rohrer, C., Mertens, S., Elhennawy, K., Ottolenghi, L., Mazur, M., Paris, S., & Schwendicke, F. (2021) · Journal of Dentistry · doi:10.1016/j.jdent.2021.103615 · Cited by 109
A rasterized building footprint dataset for the United States
Heris, M. P., Foks, N. L., Bagstad, K. J., Troy, A. R., & Ancona, Z. H. (2020) · Scientific Data · doi:10.1038/s41597-020-0542-3 · Cited by 105
Systematic Analysis and Biomarker Study for Alzheimer’s Disease
Li, X., Wang, H., Long, J., Pan, G., He, T., Anichtchik, O., Belshaw, R., Albani, D., Edison, P., Green, E. K., et al. (2018) · Scientific Reports · doi:10.1038/s41598-018-35789-3 · Cited by 87
Application of machine learning in rheumatic disease research
Kim, K., & Tagkopoulos, I. (2019) · The Korean Journal of Internal Medicine · doi:10.3904/kjim.2018.349 · Cited by 95
Xuan, P., Cao, Y., Zhang, T., Kong, R., & Zhang, Z. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00416 · Cited by 87
Barbieri, S., Kemp, J., Perez-Concha, O., Kotwal, S., Gallagher, M., Ritchie, A., & Jorm, L. (2020) · Scientific Reports · doi:10.1038/s41598-020-58053-z · Cited by 86
Liu, T., Ge, X., Yu, J., Guo, Y., Wang, Y., Wang, W., & Cui, L. (2018) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-018-1796-5 · Cited by 86
Varghese, B., Chen, F., Hwang, D., Palmer, S. L., De Castro Abreu, A. L., Ukimura, O., Aron, M., Aron, M., Gill, I., Duddalwar, V., et al. (2019) · Scientific Reports · doi:10.1038/s41598-018-38381-x · Cited by 84
A comparison of penalised regression methods for informing the selection of predictive markers
Greenwood, C. J., Youssef, G. J., Letcher, P., Macdonald, J. A., Hagg, L. J., Sanson, A. V., McIntosh, J. E., Hutchinson, D. M., Toumbourou, J. W., Fuller‐Tyszkiewicz, M., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0242730 · Cited by 116
An empirical evaluation of sampling methods for the classification of imbalanced data
Kim, M., & Hwang, K. (2022) · PLoS ONE · doi:10.1371/journal.pone.0271260 · Cited by 130
Bonikowski, B., Luo, Y., & Stuhler, O. (2022) · Sociological Methods & Research · doi:10.1177/00491241221122317 · Cited by 115
Raihan-Al-Masud, M., & Mondal, M. R. H. (2020) · PLoS ONE · doi:10.1371/journal.pone.0228422 · Cited by 99
Kusuma, K., Larsen, M. E., Quiroz, J. C., Gillies, M. B., Burnett, A., Qian, J., & Torok, M. H. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.09.050 · Cited by 101
Artificial Intelligence–enabled Decision Support in Surgery: State-of-the-art and Future Directions
Loftus, T. J., Altieri, M. S., Balch, J. A., Abbott, K. L., Choi, J., Marwaha, J. S., Hashimoto, D. A., Brat, G. A., Raftopoulos, Y., Evans, H. L., et al. (2023) · Annals of Surgery · doi:10.1097/sla.0000000000005853 · Cited by 106
Inf2vec: Latent Representation Model for Social Influence Embedding
Feng, S., Cong, G., Khan, A., Li, X., Liu, Y., & Chee, Y. M. (2018) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2018.00089 · Cited by 94
DeVries, Z., Hoda, M., Rivers, C. S., Maher, A., Wai, E. K., Moravek, D., Stratton, A. C., Kingwell, S. P., Fallah, N., Paquet, J., et al. (2019) · The Spine Journal · doi:10.1016/j.spinee.2019.09.007 · Cited by 91
Spatial fidelity of MEG/EEG source estimates: A general evaluation approach
Samuelsson, J. G., Peled, N., Mamashli, F., Ahveninen, J., & Hämäläinen, M. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117430 · Cited by 110
Pimentel, M. A. F., Redfern, O. C., Malycha, J., Meredith, P., Prytherch, D., Briggs, J., Young, J. D., Clifton, D. A., Tarassenko, L., & Watkinson, P. J. (2021) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202007-2700oc · Cited by 84
Huang, C., Li, S.-X., Caraballo, C., Masoudi, F. A., Rumsfeld, J. S., Spertus, J. A., Normand, S.-L. T., Mortazavi, B. J., & Krumholz, H. M. (2021) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.120.007526 · Cited by 83
Bonavita, I., Rafael-Palou, X., Ceresa, M., Piella, G., Ribas, V. J., & Ballester, M. Á. G. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.105172 · Cited by 98
Homeyer, A., Geißler, C., Schwen, L. O., Zakrzewski, F., Evans, T., Strohmenger, K., Westphal, M., Bülow, R. D., Kargl, M., Karjauv, A., et al. (2022) · Modern Pathology · doi:10.1038/s41379-022-01147-y · Cited by 95
Qureshi, A. S., & Roos, T. (2022) · Neural Processing Letters · doi:10.1007/s11063-022-11049-4 · Cited by 99
Introduction to Artificial Intelligence and Machine Learning for Pathology
Harrison, J. H., Gilbertson, J. R., Hanna, M. G., Olson, N. H., Seheult, J. N., Sorace, J. M., & Stram, M. N. (2021) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2020-0541-cp · Cited by 82
Credit Card Fraud Detection with Autoencoder and Probabilistic Random Forest
Lin, T.-H., & Jiang, J. (2021) · Mathematics · doi:10.3390/math9212683 · Cited by 118
On TCR binding predictors failing to generalize to unseen peptides
Grazioli, F., Mösch, A., Machart, P., Li, K., Alqassem, I., O’Donnell, T. J., & Min, M. R. (2022) · Frontiers in Immunology · doi:10.3389/fimmu.2022.1014256 · Cited by 80
Kee, O. T., Harun, H., Mustafa, N., Abdul Murad, N. A., Chin, S. F., Jaafar, R., & Abdullah, N. (2023) · Cardiovascular Diabetology · doi:10.1186/s12933-023-01741-7 · Cited by 82
“To address the imbalanced nature of clinical data, which is very common in life sciences, precision-recall curve (PRC) is the recommended metric to display the true performance of a prediction model [60].”
BackgroundIdakwo, G., Thangapandian, S., Luttrell, J., Zhou, Z., Zhang, C., & Gong, P. (2021) · doi:10.21079/11681/41302 · Cited by 82
Drug repositioning through integration of prior knowledge and projections of drugs and diseases
Xuan, P., Cao, Y., Zhang, T., Wang, X., Pan, S., & Shen, T. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz182 · Cited by 83
Evaluation of Machine Learning Classifiers for Predicting Deep Convection
Ukkonen, P., & Mäkelä, A. (2019) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2018ms001561 · Cited by 86
Vishwakarma, G., Sonpal, A., & Hachmann, J. (2021) · Trends in Chemistry · doi:10.1016/j.trechm.2020.12.004 · Cited by 101
Venn diagram analysis overestimates the extent of circadian rhythm reprogramming
Pelikan, A., Herzel, H., Kramer, A., & Ananthasubramaniam, B. (2021) · FEBS Journal · doi:10.1111/febs.16095 · Cited by 87
Sarkar, A., Maniruzzaman, M., Alahe, M. A., & Ahmad, M. (2023) · Journal of Sensors · doi:10.1155/2023/1224619 · Cited by 107
Prognostication and risk factors for cystic fibrosis via automated machine learning
Alaa, A. M., & van der Schaar, M. (2018) · Scientific Reports · doi:10.1038/s41598-018-29523-2 · Cited by 80
“…models’ precision (PPV) and recall (sensitivity): the AUC-PR is an estimate for the area under the precision-recall curve using the trapezoidal rule 44,47 , whereas the average precision is a weighted mean of precisions achieved at each threshold on the (non-interpolated) precision-recall curve,…”
MethodMauger, C., Gilbert, K., Lee, A. M., Sanghvi, M. M., Aung, N., Fung, K., Carapella, V., Piechnik, S. K., Neubauer, S., Petersen, S. E., et al. (2019) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/s12968-019-0551-6 · Cited by 98
Hernandez‐Boussard, T., Monda, K. L., Crespo, B. C., & Riskin, D. J. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz119 · Cited by 89
Drug-target interaction prediction with tree-ensemble learning and output space reconstruction
Pliakos, K., & Vens, C. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-3379-z · Cited by 89
Tuning model parameters in class‐imbalanced learning with precision‐recall curve
Fu, G., Yi, L., & Pan, J. (2018) · Biometrical Journal · doi:10.1002/bimj.201800148 · Cited by 95
Sobrinho, Á., Queiroz, A. C. M. D. S., da Silva, L. D., Costa, E. D. B., Pinheiro, M. E., & Perkusich, Â. (2020) · IEEE Access · doi:10.1109/access.2020.2971208 · Cited by 92
Planet craters detection based on unsupervised domain adaptation
Zhang, Z., Xu, Y., Song, J., Zhou, Q., Rasol, J., & Ma, L. (2023) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2023.3285512 · Cited by 105
Combining Citizen Science and Deep Learning to Amplify Expertise in Neuroimaging
Keshavan, A., Yeatman, J. D., & Rokem, A. (2019) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2019.00029 · Cited by 81
Use of machine learning techniques to identify HIV predictors for screening in sub-Saharan Africa
Charles, M., McSharry, P., Ngaruye, I., & Musabanganji, E. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01346-2 · Cited by 96
Diani, K., Amar, H., Azmi, R., Elbelrhiti, H., Nouayti, N., & Fathi, N. (2021) · Environmental Challenges · doi:10.1016/j.envc.2021.100230 · Cited by 92
Machine Learning for Understanding and Predicting Injuries in Football
Majumdar, A., Bakirov, R., Hodges, D., Scott, S., & Rees, T. (2022) · Sports Medicine - Open · doi:10.1186/s40798-022-00465-4 · Cited by 118
PET/CT based cross-modal deep learning signature to predict occult nodal metastasis in lung cancer
Zhong, Y., Cai, C., Chen, T., Gui, H., Deng, J., Yang, M., Yu, B., Song, Y., Wang, T., Sun, X., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-42811-4 · Cited by 99
Lv, Z., Liu, J., Sun, W., Lei, T., Benediktsson, J. A., & Jia, X. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3334521 · Cited by 77
The Missing Pieces of Artificial Intelligence in Medicine
Gilvary, C., Madhukar, N., Elkhader, J., & Elemento, O. (2019) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2019.06.001 · Cited by 76
The Effect of Neutral Recombination Variation on Genome Scans for Selection
Lotterhos, K. E. (2019) · G3 Genes Genomes Genetics · doi:10.1534/g3.119.400088 · Cited by 92
Using machine learning to predict opioid misuse among U.S. adolescents
Han, D., Lee, S., & Seo, D. (2019) · Preventive Medicine · doi:10.1016/j.ypmed.2019.105886 · Cited by 88
Zabihollahy, F., Schieda, N., Krishna, S., & Ukwatta, E. (2020) · European Radiology · doi:10.1007/s00330-020-06787-9 · Cited by 76
Bychkov, D., Linder, N., Tiulpin, A., Kücükel, H., Lundin, M., Nordling, S., Sihto, H., Isola, J., Lehtimäki, T., Kellokumpu-Lehtinen, P.-L., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-83102-6 · Cited by 76
Groot, O. Q., Bindels, B. J. J., Ogink, P. T., Kapoor, N. D., Twining, P. K., Collins, A. K., Bongers, M. E. R., Lans, A., Oosterhoff, J. H. F., Karhade, A. V., et al. (2021) · Acta Orthopaedica · doi:10.1080/17453674.2021.1910448 · Cited by 82
Feucherolles, M., Nennig, M., Becker, S. L., Martiny, D., Losch, S., Penny, C., Cauchie, H.-M., & Ragimbeau, C. (2022) · Frontiers in Microbiology · doi:10.3389/fmicb.2021.804484 · Cited by 74
Graph Attention Network Model with Defined Applicability Domains for Screening PBT Chemicals
Wang, H., Wang, Z., Chen, J., & Liu, W. (2022) · Environmental Science & Technology · doi:10.1021/acs.est.2c00765 · Cited by 78
Machine Learning for Endometrial Cancer Prediction and Prognostication
Bhardwaj, V., Sharma, A., Snijesh, V. P., Gul, I., Zhang, X., Lobie, P. E., Qin, P., & Pandey, V. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.852746 · Cited by 98
Deep Network-Based Comprehensive Parotid Gland Tumor Detection
Sünnetci, K. M., Kaba, E., Çeliker, F. B., & Alkan, A. (2023) · Academic Radiology · doi:10.1016/j.acra.2023.04.028 · Cited by 76
Incorporating Metric Learning and Adversarial Network for Seasonal Invariant Change Detection
Zhao, W., Mou, L., Chen, J., Bo, Y., & Emery, W. J. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2953879 · Cited by 75
Predicting breast cancer risk using interacting genetic and demographic factors and machine learning
Behravan, H., Hartikainen, J. M., Tengström, M., Kosma, V., & Mannermaa, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-66907-9 · Cited by 89
Metadata
- Authors
- Saito, T., & Rehmsmeier, M.
- Affiliations
- University of Bergen (NO)
- Corresponding author
- Saito, Takaya
- Rehmsmeier, Marc
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-03-04
- Volume
- 10
- Issue
- 3
- Pages
- e0118432
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4349800
- OpenAlex
- W1966716734
- Cited by
- 5,309
- References
- 56
- Citations by year
- 2026: 1482 · 2025: 762 · 2024: 528 · 2023: 505 · 2022: 551 · 2021: 563 · 2020: 402 · 2019: 283 · 2018: 133 · 2017: 64 · 2016: 27 · 2015: 3
- FWCI
- 74.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Universitetet i Bergen
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- class imbalance, binary classification, classifier evaluation, specificity, positive predictive value, sensitivity, bioinformatics classification, visual interpretability
- MeSH terms
- Classification, ROC Curve, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.