The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets
Open accessThe DOI matches its Crossref record; 55 of the 56 references it cites were found in a registry or an index.
Abstract
Binary classifiers are routinely evaluated with performance measures such as sensitivity and specificity, and performance is frequently illustrated with Receiver Operating Characteristics (ROC) plots. Alternative measures such as positive predictive value (PPV) and the associated Precision/Recall (PRC) plots are used less frequently. Many bioinformatics studies develop and evaluate classifiers that are to be applied to strongly imbalanced datasets in which the number of negatives outweighs the number of positives significantly. While ROC plots are visually appealing and provide an overview of a classifier's performance across a wide range of specificities, one can ask whether ROC plots could be misleading when applied in imbalanced classification scenarios. We show here that the visual interpretability of ROC plots in the context of imbalanced datasets can be deceptive with respect to conclusions about the reliability of classification performance, owing to an intuitive but wrong interpretation of specificity. PRC plots, on the other hand, can provide the viewer with an accurate prediction of future classification performance due to the fact that they evaluate the fraction of true positives among positive predictions. Our findings have potential implications for the interpretation of a large number of studies that use ROC plots on imbalanced datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“which is a more informative measure than ROC analysis in imbalanced scenarios59.”
From K-STAMM: a knowledge-enhanced spatial - temporal attention model with multimodal fusion for pneumonia prediction (2026) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Because extreme prediction is by definition an imbalanced classification problem, we base our analysis on precision-recall plots which are well-suited for this case [42].”
From Learning skillful medium-range global weather forecasting (2023) · cites this paper as [42]
In the citing paper
“Furthermore, careful consideration should be given when selecting evaluation metrics, as the precision-recall curve is more informative for imbalanced datasets than the most commonly used receiver-operating characteristic curve (true positive rate vs. false positive rate) (Fig. 5C).115”
From Use of artificial intelligence for liver diseases: A survey from the EASL congress 2024 (2024) · cites this paper as [115]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Bartel (2004) · doi:10.1016/s0092-8674(04)00045-5 · Cited by 34,714
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 3×
“Here, we review the theoretical background of commonly used evaluation measures, specifically ROC [24, 25], PRC [26], concentrated ROC (CROC) [27], and Cost Curves (CC) [28].”
BackgroundMethodThe meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
Bioconductor: open software development for computational biology and bioinformatics
Gentleman, R. C., Carey, V. J., Bates, D. M., Bolstad, B., Dettling, M., Dudoit, S., Ellis, B., Gautier, L., Ge, Y., Gentry, J., et al. (2004) · Genome Biology · doi:10.1186/gb-2004-5-10-r80 · Cited by 12,172 · cited 1×
“Several integrated machine learning and bioinformatics platforms such as WEKA [34] and Bioconductor [42, 43] also have basic functions or libraries for making ROC and PRC plots.”
MethodA training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
R: A Language for Data Analysis and Graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,220
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Cited by
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Chicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143
Hannun, A. Y., Rajpurkar, P., Haghpanahi, M., Tison, G. H., Bourn, C., Turakhia, M. P., & Ng, A. Y. (2019) · Nature Medicine · doi:10.1038/s41591-018-0268-3 · Cited by 3,148
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Classification assessment methods
Tharwat, A. (2021) · Applied Computing and Informatics · doi:10.1016/j.aci.2018.08.003 · Cited by 2,417
Learning skillful medium-range global weather forecasting
Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton-Rosen, Z., Hu, W., et al. (2023) · Science · doi:10.1126/science.adi2336 · Cited by 1,466
A clinically applicable approach to continuous prediction of future acute kidney injury
Tomašev, N., Glorot, X., Rae, J. W., Zielinski, M., Askham, H., Saraiva, A., Mottram, A., Meyer, C., Ravuri, S., Protsyuk, I., et al. (2019) · Nature · doi:10.1038/s41586-019-1390-1 · Cited by 1,086
“PR AUC is known to be more informative for class-imbalanced predictive tasks 53 , as it is more sensitive to changes in the number of false positive predictions.”
Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., & Song, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01501 · Cited by 1,090
“With these anomaly scores, we can compute the area under the precision-recall curve [43].”
MethodHendrycks, D., Zhao, K., Basart, S., Steinhardt, J., & Song, D. (2019) · arXiv · doi:10.48550/arxiv.1907.07174 · Cited by 1,058
Artificial intelligence for mental health and mental illnesses: an overview
Graham, S., Depp, C., Lee, E. E., Nebeker, C., Tu, X., Kim, H.-C., & Jeste, D. V. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1094-0 · Cited by 970
Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
Liang, S., Li, Y., & Srikant, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02690 · Cited by 611
eD octor: machine learning and the future of medicine
Handelman, G. S., Kok, H. K., Chandra, R. V., Razavi, A. H., Lee, M. J., & Asadi, H. (2018) · Journal of Internal Medicine · doi:10.1111/joim.12822 · Cited by 956
On evaluation metrics for medical applications of artificial intelligence
Hicks, S. A., Strümke, I., Thambawita, V., Hammou, M., Riegler, M. A., Halvorsen, P., & Parasa, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-09954-8 · Cited by 934
Automatic diagnosis of the 12-lead ECG using a deep neural network
Ribeiro, A. H., Ribeiro, M. H., Paixão, G. M. M., Oliveira, D. M., Gomes, P. R., Canazart, J. A., Ferreira, M. P. S., Andersson, C. R., Macfarlane, P. W., Meira, W., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-15432-4 · Cited by 894
Chicco, D., Tötsch, N., & Jurman, G. (2021) · BioData Mining · doi:10.1186/s13040-021-00244-z · Cited by 893
Ten quick tips for machine learning in computational biology
Chicco, D. (2017) · BioData Mining · doi:10.1186/s13040-017-0155-3 · Cited by 878
Valavi, R., Guillera‐Arroita, G., Lahoz‐Monfort, J. J., & Elith, J. (2022) · Ecological Monographs · doi:10.1002/ecm.1486 · Cited by 830
Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data
Pratapa, A., Jalihal, A. P., Law, J. N., Bharadwaj, A., & Murali, T. M. (2020) · Nature Methods · doi:10.1038/s41592-019-0690-6 · Cited by 792
Ribeiro, A. H., Ribeiro, M. H., Paixao, G. M., Oliveira, D. M., Gomes, P. R., Canazart, J. A., Ferreira, M. P. S., Andersson, C. R., Macfarlane, P. W., Meira, W., et al. (2020) · RePEc: Research Papers in Economics · doi:10.5281/zenodo.3625006 · Cited by 715
SPot-the-Difference Self-supervised Pre-training for Anomaly Detection and Segmentation
Zou, Y., Jeong, J., Pemula, L., Zhang, D., & Dabeer, O. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20056-4_23 · Cited by 651
Classification Model Evaluation Metrics
Vujovic, Ž. Ð. (2021) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2021.0120670 · Cited by 620
Chicco, D., & Jurman, G. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-1023-5 · Cited by 596
Benchmarking Metagenomics Tools for Taxonomic Classification
Ye, S. H., Siddle, K. J., Park, D. J., & Sabeti, P. C. (2019) · Cell · doi:10.1016/j.cell.2019.07.010 · Cited by 593
Link prediction techniques, applications, and performance: A survey
Kumar, A., Singh, S. S., Singh, K., & Biswas, B. (2020) · Physica A: Statistical Mechanics and its Applications · doi:10.1016/j.physa.2020.124289 · Cited by 516
Automated assessment of psychiatric disorders using speech: A systematic review
Low, D. M., Bentley, K. H., & Ghosh, S. S. (2020) · Laryngoscope Investigative Otolaryngology · doi:10.1002/lio2.354 · Cited by 509
Boosting methods for multi-class imbalanced data classification: an experimental review
Tanha, J., Abdi, Y., Samadi, N., Razzaghi, N., & Asadpour, M. (2020) · Journal of Big Data · doi:10.1186/s40537-020-00349-y · Cited by 452
Identification of transcription factor binding sites using ATAC-seq
Li, Z., Schulz, M. H., Look, T., Begemann, M., Zenke, M., & Costa, I. G. (2019) · Genome Biology · doi:10.1186/s13059-019-1642-2 · Cited by 452
Role of artificial intelligence in patient safety outcomes: systematic literature review
Choudhury, A., & Asan, O. (2020) · JMIR Medical Informatics · doi:10.2196/18599 · Cited by 440
Multi-head CNN–RNN for multi-time series anomaly detection: An industrial case study
Canizo, M., Triguero, I., Conde, A., & Onieva, E. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.07.034 · Cited by 420
The area under the precision‐recall curve as a performance metric for rare binary events
Sofaer, H. R., Hoeting, J. A., & Jarnevich, C. S. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13140 · Cited by 375
IDRiD: Diabetic Retinopathy – Segmentation and Grading Challenge
Porwal, P., Pachade, S., Kokare, M., Deshmukh, G., Son, J., Bae, W., Liu, L., Wang, J., Liu, X., Gao, L., et al. (2020) · Medical Image Analysis · doi:10.1016/j.media.2019.101561 · Cited by 372
Artificial intelligence in dental research: Checklist for authors, reviewers, readers
Schwendicke, F., Singh, T., Lee, J.-H., Gaudin, R., Chaurasia, A., Wiegand, T., Uribe, S., & Krois, J. (2021) · Journal of Dentistry · doi:10.1016/j.jdent.2021.103610 · Cited by 360
Selenium Deficiency Is Associated with Mortality Risk from COVID-19
Moghaddam, A., Heller, R., Sun, Q., Seelig, J., Cherkezov, A., Seibert, L., Hackler, J., Seemann, P., Diegmann, J., Pilz, M., et al. (2020) · Nutrients · doi:10.3390/nu12072098 · Cited by 359
Cen, L.-P., Ji, J., Lin, J.-W., Ju, S.-T., Lin, H.-J., Li, T.-P., Wang, Y., Yang, J.-F., Liu, Y.-F., Tan, S., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-25138-w · Cited by 346
Carrington, A. M., Manuel, D. G., Fieguth, P. W., Ramsay, T., Osmani, V., Wernly, B., Bennett, C., Hawken, S., Magwood, O., Sheikh, Y., et al. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3145392 · Cited by 339
ROC and AUC with a Binary Predictor: a Potentially Misleading Metric
Muschelli, J. (2020) · Journal of Classification · doi:10.1007/s00357-019-09345-1 · Cited by 293
The impact of site-specific digital histology signatures on deep learning model accuracy and bias
Howard, F. M., Dolezal, J., Kochanny, S., Schulte, J., Chen, H., Heij, L., Huo, D., Nanda, R., Olopade, O. I., Kather, J. N., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-24698-1 · Cited by 330
Maxwell, A. E., Warner, T. A., & Guillén, L. A. (2021) · Remote Sensing · doi:10.3390/rs13132450 · Cited by 327
Jiao, Y., & Du, P. (2016) · Quantitative Biology · doi:10.1007/s40484-016-0081-2 · Cited by 307
A systematic survey of computer-aided diagnosis in medicine: Past and present developments
Yanase, J., & Triantaphyllou, E. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112821 · Cited by 303
“Furthermore, the seminal paper [Saito and Rehmsmeier, 2015a] describes some important issues related to the plotting of PR curves and also some domination issues between the ROC and PR curves of the same dataset.”
BackgroundDeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection
Zhang, X., Li, S., Li, X., Huang, P., Shan, J., & Chen, T. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00381 · Cited by 287
Photoplethysmography based atrial fibrillation detection: a review
Pereira, T., Tran, N., Gadhoumi, K., Pelter, M. M., Do, D. H., Lee, R. J., Colorado, R., Meisel, K., & Hu, X. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0207-9 · Cited by 300
“low prevalence).(40) In general, any reported performance metric should take into account the low prevalence of AF and be evaluated on an independent test dataset.”
BackgroundSystematic comparison of variant calling pipelines using gold standard personal exome variants
Hwang, S., Kim, E., Lee, I., & Marcotte, E. M. (2015) · Scientific Reports · doi:10.1038/srep17875 · Cited by 297
Haimovich, A. D., Ravindra, N. G., Stoytchev, S., Young, H. P., Wilson, F. P., van Dijk, D., Schulz, W. L., & Taylor, R. A. (2020) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2020.07.022 · Cited by 294
Desai, R. J., Wang, S. V., Vaduganathan, M., Evers, T., & Schneeweiss, S. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.18962 · Cited by 292
Qin, Z. Z., Ahmed, S., Sarker, M. S., Paul, K., Adel, A. S. S., Naheyan, T., Barrett, R., Banu, S., & Creswell, J. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00116-3 · Cited by 290
Noguchi, Y., Tachi, T., & Teramachi, H. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab347 · Cited by 285
Smedley, D., Schubach, M., Jacobsen, J. O. B., Köhler, S., Zemojtel, T., Spielmann, M., Jäger, M., Hochheiser, H., Washington, N. L., McMurry, J. A., et al. (2016) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2016.07.005 · Cited by 283
Evaluating the analytical validity of circulating tumor DNA sequencing assays for precision oncology
Deveson, I. W., Gong, B., Lai, K., LoCoco, J. S., Richmond, T. A., Schageman, J., Zhang, Z., Novoradovskaya, N., Willey, J. C., Jones, W., et al. (2021) · Nature Biotechnology · doi:10.1038/s41587-021-00857-z · Cited by 276
A deep learning method for predicting metabolite–disease associations via graph neural network
Sun, F., Sun, J., & Zhao, Q. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac266 · Cited by 272
Precrec: fast and accurate precision–recall and ROC curve calculations in R
Saito, T., & Rehmsmeier, M. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btw570 · Cited by 267
Self-citationKim, J. I., Maguire, F., Tsang, K. K., Gouliouris, T., Peacock, S. J., McAllister, T. A., McArthur, A. G., & Beiko, R. G. (2022) · Clinical Microbiology Reviews · doi:10.1128/cmr.00179-21 · Cited by 256
Imani, M., Beikmohammadi, A., & Arabnia, H. R. (2025) · Technologies · doi:10.3390/technologies13030088 · Cited by 256
Steven Eyobu, O., & Han, D. S. (2018) · Sensors · doi:10.3390/s18092892 · Cited by 259
“This phenomenon is common in binary classification [19,20] where only two classes are involved.”
BackgroundVeReMi: A Dataset for Comparable Evaluation of Misbehavior Detection in VANETs
van der Heijden, R. W., Lukaseder, T., & Kargl, F. (2018) · Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-01701-9_18 · Cited by 252
Development and Validation of the Phoenix Criteria for Pediatric Sepsis and Septic Shock
Sanchez-Pinto, L. N., Bennett, T. D., DeWitt, P. E., Russell, S., Rebull, M. N., Martin, B., Akech, S., Albers, D. J., Alpern, E. R., Balamuth, F., et al. (2024) · JAMA · doi:10.1001/jama.2024.0196 · Cited by 241
Ayala Solares, J. R., Diletta Raimondi, F. E., Zhu, Y., Rahimian, F., Canoy, D., Tran, J., Pinho Gomes, A. C., Payberah, A. H., Zottoli, M., Nazarzadeh, M., et al. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103337 · Cited by 247
“However, this 341 metric can be misleading when the classes are very imbalanced [74, 75, 76].”
BackgroundNetwork Intrusion Detection Based on PSO-Xgboost Model
Jiang, H., He, Z., Ye, G., & Zhang, H. (2020) · IEEE Access · doi:10.1109/access.2020.2982418 · Cited by 243
An Expanded Obstetric Comorbidity Scoring System for Predicting Severe Maternal Morbidity
Leonard, S. A., Kennedy, C. J., Carmichael, S. L., Lyell, D. J., & Main, E. K. (2020) · Obstetrics & Gynecology · doi:10.1097/aog.0000000000004022 · Cited by 240
Walch, O., Huang, Y., Forger, D., & Goldstein, C. (2019) · Sleep · doi:10.1093/sleep/zsz180 · Cited by 240
“In this case, the recall (on the x -axis) is the fraction of wake epochs scored correctly, and the precision (on the y -axis) shows the fraction of all epochs labeled wake that were truly wake [39].”
BackgroundA community approach to mortality prediction in sepsis via gene expression analysis
Sweeney, T. E., Perumal, T. M., Henao, R., Nichols, M., Howrylak, J. A., Choi, A. M., Bermejo-Martin, J. F., Almansa, R., Tamayo, E., Davenport, E. E., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-03078-2 · Cited by 237
Precision–recall curve (PRC) classification trees
Miao, J., & Zhu, W. (2022) · Evolutionary Intelligence · doi:10.1007/s12065-021-00565-2 · Cited by 232
Tran, T. T. V., Surya Wibowo, A., Tayara, H., & Chong, K. T. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c00200 · Cited by 226
Early Prediction of Sepsis in the ICU Using Machine Learning: A Systematic Review
Moor, M., Rieck, B., Horn, M., Jutzeler, C. R., & Borgwardt, K. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.607952 · Cited by 231
Lo-Ciganic, W.-H., Huang, J. L., Zhang, H. H., Weiss, J. C., Wu, Y., Kwoh, C. K., Donohue, J. M., Cochran, G., Gordon, A. J., Malone, D. C., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.0968 · Cited by 229
Reference standards for next-generation sequencing
Hardwick, S. A., Deveson, I. W., & Mercer, T. R. (2017) · Nature Reviews Genetics · doi:10.1038/nrg.2017.44 · Cited by 225
Machine Learning in Epidemiology and Health Outcomes Research
Wiemken, T. L., & Kelley, R. R. (2020) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040119-094437 · Cited by 222
Anomaly Detection of Time Series With Smoothness-Inducing Sequential Variational Auto-Encoder
Li, L., Yan, J., Wang, H., & Jin, Y. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2980749 · Cited by 214
MiM-ISTD: Mamba-in-Mamba for Efficient Infrared Small-Target Detection
Chen, T., Ye, Z., Tan, Z., Gong, T., Wu, Y., Chu, Q., Liu, B., Yu, N., & Ye, J. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3485721 · Cited by 214
Nilsson, S. R., Goodwin, N. L., Choong, J. J., Hwang, S., Wright, H. R., Norville, Z. C., Tong, X., Lin, D., Bentzley, B. S., Eshel, N., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.04.19.049452 · Cited by 212
A systematic study of key elements underlying molecular property prediction
Deng, J., Yang, Z., Wang, H., Ojima, I., Samaras, D., & Wang, F. (2023) · Nature Communications · doi:10.1038/s41467-023-41948-6 · Cited by 213
Machine learning and applications in microbiology
Goodswen, S. J., Barratt, J. L. N., Kennedy, P. J., Kaufer, A., Calarco, L., & Ellis, J. T. (2021) · FEMS Microbiology Reviews · doi:10.1093/femsre/fuab015 · Cited by 209
Csillik, O., Cherbini, J., Johnson, R., Lyons, A., & Kelly, M. (2018) · Drones · doi:10.3390/drones2040039 · Cited by 210
Benchmarking Computational Doublet-Detection Methods for Single-Cell RNA Sequencing Data
Xi, N. M., & Li, J. J. (2021) · Cell Systems · doi:10.1016/j.cels.2020.11.008 · Cited by 208
Zhao, J., Feng, Q., Wu, P., Lupu, R. A., Wilke, R. A., Wells, Q. S., Denny, J. C., & Wei, W.-Q. (2019) · Scientific Reports · doi:10.1038/s41598-018-36745-x · Cited by 209
The receiver operating characteristic curve accurately assesses imbalanced datasets
Richardson, E., Trevizani, R., Greenbaum, J. A., Carter, H., Nielsen, M., & Peters, B. (2024) · Patterns · doi:10.1016/j.patter.2024.100994 · Cited by 200
Burke, T. A., Ammerman, B. A., & Jacobucci, R. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.11.073 · Cited by 205
Quang, D., & Xie, X. (2019) · Methods · doi:10.1016/j.ymeth.2019.03.020 · Cited by 204
Credit card fraud detection in the era of disruptive technologies: A systematic review
Cherif, A., Badhib, A., Ammar, H., Alshehri, S., Kalkatawi, M., & Imine, A. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.11.008 · Cited by 200
Biomedical data and computational models for drug repositioning: a comprehensive review
Luo, H., Li, M., Yang, M., Wu, F.-X., Li, Y., & Wang, J. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbz176 · Cited by 198
A Generative Deep Learning Approach to Stochastic Downscaling of Precipitation Forecasts
Harris, L., McRae, A. T. T., Chantry, M., Dueben, P. D., & Palmer, T. N. (2022) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2022ms003120 · Cited by 198
Lau, E. T., Sun, L., & Yang, Q. (2019) · SN Applied Sciences · doi:10.1007/s42452-019-0884-7 · Cited by 196
A baseline for detecting misclassified and out-of-distribution examples in neural networks
Hendrycks, D. (2025) · arXiv (Cornell University) · doi:10.57702/9gs720u1 · Cited by 196
Structure-aware protein–protein interaction site prediction using deep graph convolutional network
Yuan, Q., Chen, J., Zhao, H., Zhou, Y., & Yang, Y. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab643 · Cited by 193
DeepMicro: Deep representation learning for disease prediction based on microbiome data
Oh, M., & Zhang, L. (2020) · Scientific Reports · doi:10.1038/s41598-020-63159-5 · Cited by 190
Comprehensive Overview of Bottom-Up Proteomics Using Mass Spectrometry
Jiang, Y., Rex, D. A. B., Schuster, D., Neely, B. A., Rosano, G. L., Volkmar, N., Momenzadeh, A., Peters-Clarke, T. M., Egbert, S. B., Kreimer, S., et al. (2024) · ACS Measurement Science Au · doi:10.1021/acsmeasuresciau.3c00068 · Cited by 190
SQuIRE reveals locus-specific regulation of interspersed repeat expression
Yang, W. R., Ardeljan, D., Pacyna, C. N., Payer, L. M., & Burns, K. H. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky1301 · Cited by 187
Security issues in Internet of Vehicles (IoV): A comprehensive survey
Taslimasa, H., Dadkhah, S., Neto, E. C. P., Xiong, P., Ray, S., & Ghorbani, A. A. (2023) · Internet of Things · doi:10.1016/j.iot.2023.100809 · Cited by 185
Foody, G. M. (2023) · PLOS ONE · doi:10.1371/journal.pone.0291908 · Cited by 186
The False positive problem of automatic bot detection in social science research
Rauchfleisch, A., & Kaiser, J. (2020) · PLOS ONE · doi:10.1371/journal.pone.0241045 · Cited by 184
“Our analysis also shows what Saito and Rehmsmeier [33] recommend in their study: the PR-curve is more informative if we are dealing with imbalanced data in the population.”
BackgroundResultInfluentialPredicting and explaining employee turnover intention
Lazzari, M., Alvarez, J. M., & Ruggieri, S. (2022) · International Journal of Data Science and Analytics · doi:10.1007/s41060-022-00329-w · Cited by 181
Real-World Anomaly Detection by Using Digital Twin Systems and Weakly Supervised Learning
Castellani, A., Schmitt, S., & Squartini, S. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3019788 · Cited by 181
“In the situation with highly imbalanced class sizes, it was shown that the PRC is more informative than ROC [27], since it better reflects the correct prediction of the minority class.”
InfluentialFan, X., Wang, X., Zhang, X., & Yu, X. (. (2022) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.108185 · Cited by 179
Park, Y. W., Oh, J., You, S. C., Han, K., Ahn, S. S., Choi, Y. S., Chang, J. H., Kim, S. H., & Lee, S.-K. (2019) · European Radiology · doi:10.1007/s00330-018-5830-3 · Cited by 177
Deep learning in pharmacogenomics: from gene regulation to patient stratification
Kalinin, A. A., Higgins, G. A., Reamaroon, N., Soroushmehr, S., Allyn-Feuer, A., Dinov, I. D., Najarian, K., & Athey, B. D. (2018) · Pharmacogenomics · doi:10.2217/pgs-2018-0008 · Cited by 207
Li, Z., Huang, M., Liu, G., & Jiang, C. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114750 · Cited by 175
MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare
Choi, E., Xiao, C., Stewart, W. F., & Sun, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.09593 · Cited by 157
Wang, R., Jiang, Y., Jin, J., Yin, C., Yu, H., Wang, F., Feng, J., Su, R., Nakai, K., Zou, Q., et al. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkad055 · Cited by 176
“The PR curve is used more frequently 235 to evaluate the performance of a model on imbalanced datasets (61).”
MethodHeyndrickx, W., Mervin, L., Morawietz, T., Sturm, N., Friedrich, L., Zalewski, A., Pentina, A., Humbeck, L., Oldenhof, M., Niwayama, R., et al. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c00799 · Cited by 170
Early Methods for Detecting Adversarial Images
Hendrycks, D., & Gimpel, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.00530 · Cited by 171
“The baseline detector has an AUPR equal to the precision (Saito & Rehmsmeier, 2015), and a “perfect” classifier has an AUPR of 100% .”
Method
Metadata
- Authors
- Saito, T., & Rehmsmeier, M.
- Affiliations
- University of Bergen (NO)
- Corresponding author
- Saito, Takaya
- Rehmsmeier, Marc
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-03-04
- Volume
- 10
- Issue
- 3
- Pages
- e0118432
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4349800
- OpenAlex
- W1966716734
- Cited by
- 5,309
- References
- 56
- Citations by year
- 2026: 1482 · 2025: 762 · 2024: 528 · 2023: 505 · 2022: 551 · 2021: 563 · 2020: 402 · 2019: 283 · 2018: 133 · 2017: 64 · 2016: 27 · 2015: 3
- FWCI
- 74.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Universitetet i Bergen
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- class imbalance, binary classification, classifier evaluation, specificity, positive predictive value, sensitivity, bioinformatics classification, visual interpretability
- MeSH terms
- Classification, ROC Curve, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.