The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets
Open accessThe DOI matches its Crossref record; 55 of the 56 references it cites were found in a registry or an index.
Abstract
Binary classifiers are routinely evaluated with performance measures such as sensitivity and specificity, and performance is frequently illustrated with Receiver Operating Characteristics (ROC) plots. Alternative measures such as positive predictive value (PPV) and the associated Precision/Recall (PRC) plots are used less frequently. Many bioinformatics studies develop and evaluate classifiers that are to be applied to strongly imbalanced datasets in which the number of negatives outweighs the number of positives significantly. While ROC plots are visually appealing and provide an overview of a classifier's performance across a wide range of specificities, one can ask whether ROC plots could be misleading when applied in imbalanced classification scenarios. We show here that the visual interpretability of ROC plots in the context of imbalanced datasets can be deceptive with respect to conclusions about the reliability of classification performance, owing to an intuitive but wrong interpretation of specificity. PRC plots, on the other hand, can provide the viewer with an accurate prediction of future classification performance due to the fact that they evaluate the fraction of true positives among positive predictions. Our findings have potential implications for the interpretation of a large number of studies that use ROC plots on imbalanced datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“which is a more informative measure than ROC analysis in imbalanced scenarios59.”
From K-STAMM: a knowledge-enhanced spatial - temporal attention model with multimodal fusion for pneumonia prediction (2026) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Because extreme prediction is by definition an imbalanced classification problem, we base our analysis on precision-recall plots which are well-suited for this case [42].”
From Learning skillful medium-range global weather forecasting (2023) · cites this paper as [42]
In the citing paper
“Furthermore, careful consideration should be given when selecting evaluation metrics, as the precision-recall curve is more informative for imbalanced datasets than the most commonly used receiver-operating characteristic curve (true positive rate vs. false positive rate) (Fig. 5C).115”
From Use of artificial intelligence for liver diseases: A survey from the EASL congress 2024 (2024) · cites this paper as [115]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Bartel (2004) · doi:10.1016/s0092-8674(04)00045-5 · Cited by 34,714
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 3×
“Here, we review the theoretical background of commonly used evaluation measures, specifically ROC [24, 25], PRC [26], concentrated ROC (CROC) [27], and Cost Curves (CC) [28].”
BackgroundMethodThe meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
Bioconductor: open software development for computational biology and bioinformatics
Gentleman, R. C., Carey, V. J., Bates, D. M., Bolstad, B., Dettling, M., Dudoit, S., Ellis, B., Gautier, L., Ge, Y., Gentry, J., et al. (2004) · Genome Biology · doi:10.1186/gb-2004-5-10-r80 · Cited by 12,172 · cited 1×
“Several integrated machine learning and bioinformatics platforms such as WEKA [34] and Bioconductor [42, 43] also have basic functions or libraries for making ROC and PRC plots.”
MethodA training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
R: A Language for Data Analysis and Graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,220
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Cited by
Items 401–500 of 1,023
Lee, J., Jun, D. W., Song, I., & Kim, Y. (2024) · Journal of Cheminformatics · doi:10.1186/s13321-024-00808-1 · Cited by 69
Identification of dysregulated miRNAs in triple negative breast cancer: A meta‐analysis approach
Naorem, L. D., Muthaiyan, M., & Venkatesan, A. (2018) · Journal of Cellular Physiology · doi:10.1002/jcp.27839 · Cited by 61
Orlenko, A., Kofink, D., Lyytikäinen, L., Nikus, K. C., Mishra, P. P., Kuukasjärvi, P., Karhunen, P. J., Kähönen, M. A. P., Laurikka, J. O., Lehtimäki, T. J., et al. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz796 · Cited by 61
On Model Evaluation Under Non-constant Class Imbalance
Brabec, J., Komárek, T., Franc, V., & Machlica, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50423-6_6 · Cited by 80
Navigating the metric maze: a taxonomy of evaluation metrics for anomaly detection in time series
Sørbø, S., & Ruocco, M. (2024) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00988-8 · Cited by 58
“14, where the event-based metrics prioritize the cluster of three events over the single long one.”
Nationwide prediction of type 2 diabetes comorbidities
Dworzynski, P., Aasbrenn, M., Rostgaard, K., Melbye, M., Gerds, T. A., Hjalgrim, H., & Pers, T. H. (2020) · Scientific Reports · doi:10.1038/s41598-020-58601-7 · Cited by 58
Machine learning for initial insulin estimation in hospitalized patients
Nguyen, M., Jankovic, I., Kalesinskas, L., Baiocchi, M., & Chen, J. H. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab099 · Cited by 65
Polanco, E. R., Moustafa, T. E., Butterfield, A., Scherer, S. D., Cortes-Sanchez, E., Bodily, T. A., Spike, B. T., Welm, B. E., Bernard, P. S., & Zangle, T. A. (2022) · Communications Biology · doi:10.1038/s42003-022-03759-1 · Cited by 60
Hirvasniemi, J., Runhaar, J., van der Heijden, R. A., Zokaeinikoo, M., Yang, M., Li, X., Tan, J., Rajamohan, H. R., Zhou, Y., Deniz, C. M., et al. (2022) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2022.10.001 · Cited by 67
Fraud detection with natural language processing
Boulieris, P., Pavlopoulos, J., Xenos, A., & Vassalos, V. (2023) · Machine Learning · doi:10.1007/s10994-023-06354-5 · Cited by 89
Zhu, Y., Bi, D., Saunders, M. R., & Ji, Y. (2023) · Scientific Reports · doi:10.1038/s41598-023-49271-2 · Cited by 87
Skeletal Video Anomaly Detection Using Deep Learning: Survey, Challenges, and Future Directions
Mishra, P. K., Mihailidis, A., & Khan, S. S. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3358103 · Cited by 80
MLP-Net: Multilayer Perceptron Fusion Network for Infrared Small Target Detection
Wang, Z., Wang, C., Li, X., Xia, C., & Xu, J. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3515648 · Cited by 65
Automatically classifying posts into question categories on stack overflow
Beyer, S., Macho, C., Pinzger, M., & Penta, M. D. (2018) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3196321.3196333 · Cited by 74
Scantlebury, J., Brown, N., Von Delft, F., & Deane, C. M. (2020) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.0c00263 · Cited by 57
Benchmarking network propagation methods for disease gene identification
Picart‐Armada, S., Barrett, S. J., Willé, D. R., Perera-Lluna, A., Gutteridge, A., & Dessailly, B. H. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007276 · Cited by 66
Performance of quantitative measures of multimorbidity: a population-based retrospective analysis
Vela, E., Clèries, M., Monterde, D., Carot-Sans, G., Coca, M., Valero-Bover, D., Piera-Jiménez, J., García‐Eroles, L., & Sust, P. P. (2021) · BMC Public Health · doi:10.1186/s12889-021-11922-2 · Cited by 56
Maggipinto, M., Beghi, A., & Susto, G. A. (2022) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2022.3141186 · Cited by 58
Balbinot, G. C., Li, G., Kalsi‐Ryan, S., Abel, R. F., Maier, D. D., Kalke, Y.-B., Weidner, N., Rupp, R., Schubert, M. J. W., Curt, A. E. P., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-36390-7 · Cited by 67
Gao, T., & Wu, Y. (2025) · ACS Omega · doi:10.1021/acsomega.5c08467 · Cited by 66
Boehm, K. M., Bhinder, B., Raja, V. J., Dephoure, N., & Elemento, O. (2019) · BMC Bioinformatics · doi:10.1186/s12859-018-2561-z · Cited by 58
Euh, S., Lee, H., Kim, D., & Hwang, D. (2020) · IEEE Access · doi:10.1109/access.2020.2986014 · Cited by 65
Naseer, S., Ali, R. F., Khan, Y. D., & Dominic, P. D. D. (2021) · Journal of Biomolecular Structure and Dynamics · doi:10.1080/07391102.2021.1962738 · Cited by 59
Anomaly detection in IR images of PV modules using supervised contrastive learning
Bommes, L., Hoffmann, M., Buerhop‐Lutz, C., Pickel, T., Hauch, J., Brabec, C. J., Maier, A., & Peters, I. M. (2022) · Progress in Photovoltaics Research and Applications · doi:10.1002/pip.3518 · Cited by 75
Zhang, X., Akber, M. Z., & Zheng, W. (2022) · Journal of Building Engineering · doi:10.1016/j.jobe.2022.104997 · Cited by 66
High-performance automated abstract screening with large language model ensembles
Sanghera, R., Thirunavukarasu, A. J., El Khoury, M., O’Logbon, J., Chen, Y., Watt, A., Mahmood, M., Butt, H., Nishimura, G., & Soltan, A. A. S. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf050 · Cited by 52
Data Preprocessing and Feature Engineering for Data Mining: Techniques, Tools, and Best Practices
Koukaras, P., & Tjortjis, C. (2025) · AI · doi:10.3390/ai6100257 · Cited by 126
“Beyond accuracy, calibration and ROC-AUC should be reported to ensure that preprocessing choices align with the domain goals [73,84].”
Koziol, U., Koziol, M., Preza, M., Costábile, A., Brehm, K., & Castillo, E. (2016) · International Journal for Parasitology · doi:10.1016/j.ijpara.2016.05.007 · Cited by 62
Ijaq, J., Malik, G., Kumar, A., Das, P. S., Meena, N., Bethi, N., Sundararajan, V. S., & Suravajhala, P. (2019) · BMC Bioinformatics · doi:10.1186/s12859-018-2554-y · Cited by 67
Passenger mutations accurately classify human tumors
Salvadores, M., Mas-Ponte, D., & Supek, F. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006953 · Cited by 64
NeoMutate: an ensemble machine learning framework for the prediction of somatic mutations in cancer
Anzar, I., Sverchkova, A., Stratford, R., & Clancy, T. (2019) · BMC Medical Genomics · doi:10.1186/s12920-019-0508-5 · Cited by 60
Idakwo, G., Thangapandian, S., Luttrell, J., Zhou, Z., Zhang, C., & Gong, P. (2019) · Frontiers in Physiology · doi:10.3389/fphys.2019.01044 · Cited by 55
Learning Robust Deep State Space for Unsupervised Anomaly Detection in Contaminated Time-Series
Li, L., Yan, J., Wen, Q., Jin, Y., & Yang, X. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3171562 · Cited by 55
Luo, X.-Q., Kang, Y., Duan, S.-B., Yan, P., Song, G.-B., Zhang, N.-Y., Yang, S., Li, J.-X., & Zhang, H. (2023) · Journal of Medical Internet Research · doi:10.2196/41142 · Cited by 63
Cheng, Y., Gadd, D. A., Gieger, C., Monterrubio‐Gómez, K., Zhang, Y., Berta, I., Stam, M. J., Szlachetka, N., Lobzaev, E., Wrobel, N., et al. (2023) · Nature Aging · doi:10.1038/s43587-023-00391-4 · Cited by 65
Identification of DNA motifs that regulate DNA methylation
Wang, M., Zhang, K., Ngo, V., Liu, C., Fan, S., Whitaker, J. W., Chen, Y., Ai, R., Chen, Z., Wang, J., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz483 · Cited by 62
Bidelman, G. M., Mahmud, M. S., Yeasin, M., Shen, D., Arnott, S. R., & Alain, C. (2019) · Brain Structure and Function · doi:10.1007/s00429-019-01922-9 · Cited by 63
A unique chromatin profile defines adaptive genomic regions in a fungal plant pathogen
Cook, D. E., Kramer, H. M., Torres, D. E., Seidl, M. F., & Thomma, B. P. H. J. (2020) · eLife · doi:10.7554/elife.62208 · Cited by 56
Datta, S., Loftus, T. J., Ruppert, M. M., Giordano, C. R., Upchurch, G. R., Rashidi, P., Ozrazgat‐Baslanti, T., & Bihorac, A. (2020) · Journal of Surgical Research · doi:10.1016/j.jss.2020.05.007 · Cited by 63
Automated and accurate segmentation of leaf venation networks via deep learning
Xu, H., Blonder, B. W., Jodra, M., Malhi, Y. S., & Fricker, M. D. (2020) · New Phytologist · doi:10.1111/nph.16923 · Cited by 61
Artificial Intelligence in Cardiac MRI: Is Clinical Adoption Forthcoming?
Fotaki, A., Anton, E. P., Chiribiri, A., Botnar, R. M., Pushparajah, K., & Prieto, C. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2021.818765 · Cited by 65
Zhan, Y., Liu, W., & Maruyama, Y. (2022) · Remote Sensing · doi:10.3390/rs14041002 · Cited by 60
Wilimitis, D., Turer, R. W., Ripperger, M. A., McCoy, A. B., Sperry, S. H., Fielstein, E. M., Kurz, T. L., & Walsh, C. G. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.12095 · Cited by 53
Diagnosis of Crohn’s disease and ulcerative colitis using the microbiome
Kang, D.-Y., Park, J., Yeo, M., Kang, S., Kim, J., Kim, J., & Kim, S. (2023) · BMC Microbiology · doi:10.1186/s12866-023-03084-5 · Cited by 69
Mohammed, R. A., Wong, K. W., Shiratuddin, M. F., & Wang, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-97310-4_27 · Cited by 56
Predicting Kinase Inhibitor Resistance: Physics-Based and Data-Driven Approaches
Aldeghi, M., Gapsys, V., & de Groot, B. L. (2019) · ACS Central Science · doi:10.1021/acscentsci.9b00590 · Cited by 54
Predicting Drug-Target Interactions With Multi-Label Classification and Label Partitioning
Pliakos, K., Vens, C., & Tsoumakas, G. (2019) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2019.2951378 · Cited by 58
Social Network Analytics for Supervised Fraud Detection in Insurance
Óskarsdóttir, M., Ahmed, W., Antonio, K., Baesens, B., Dendievel, R., Donas, T., & Reynkens, T. (2021) · Risk Analysis · doi:10.1111/risa.13693 · Cited by 82
Obstetric comorbidity scores and disparities in severe maternal morbidity across marginalized groups
Leonard, S. A., Main, E. K., Lyell, D. J., Carmichael, S. L., Kennedy, C. J., Johnson, C., & Mujahid, M. S. (2021) · American Journal of Obstetrics & Gynecology MFM · doi:10.1016/j.ajogmf.2021.100530 · Cited by 58
Fighting Money Laundering With Statistics and Machine Learning
Jensen, R. I. T., & Iosifidis, A. (2023) · IEEE Access · doi:10.1109/access.2023.3239549 · Cited by 94
Jiang, S., Zhou, Y., Xu, W., Xia, Q. X., Yi, M., & Cheng, X. (2024) · Chemical Engineering Journal · doi:10.1016/j.cej.2024.150297 · Cited by 55
Lo, R. S. L., Leung, L. Y., Brabrand, M., Yeung, C. Y., Chan, S. Y., Lam, C., Hung, K. K. C., & Graham, C. A. (2019) · Journal of Clinical Medicine · doi:10.3390/jcm8010061 · Cited by 80
Cho, H.-M., Park, H., Dong, S.-Y., & Youn, I. (2019) · Sensors · doi:10.3390/s19204408 · Cited by 53
Jones, R. M., McMahon, D., & Hynynen, K. (2020) · Theranostics · doi:10.7150/thno.47281 · Cited by 62
Hunt, J., Rosser, D. M., & Rowe, S. P. (2020) · Journal of Accounting and Public Policy · doi:10.1016/j.jaccpubpol.2020.106785 · Cited by 62
Maarseveen, T., Meinderink, T., Reinders, M., Knitza, J., Huizinga, T., Kleyer, A., Simon, D., van den Akker, E. B., & Knevel, R. (2020) · JMIR Medical Informatics · doi:10.2196/23930 · Cited by 49
Identification of Transcription Factor Binding Sites using ATAC-seqPreprint
Li, Z., Schulz, M. H., Zenke, M., & Costa, I. G. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/362863 · Cited by 57
Anterior nucleus of the thalamus seizure detection in ambulatory humans
Gregg, N. M., Marks, V. S., Sladky, V., Lundstrom, B. N., Klassen, B. T., Messina, S. A., Brinkmann, B. H., Miller, K. J., Gompel, J. J. V., Křemen, V., et al. (2021) · Epilepsia · doi:10.1111/epi.17047 · Cited by 52
Bowd, C., Belghith, A., Zangwill, L. M., Christopher, M. A., Goldbaum, M. H., Fan, R., Rezapour, J., Moghimi, S., Kamalipour, A., Hou, H., et al. (2021) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2021.11.008 · Cited by 58
Detecting Glaucoma in the Ocular Hypertension Study Using Deep Learning
Fan, R., Bowd, C., Christopher, M. A., Brye, N., Proudfoot, J. A., Rezapour, J., Belghith, A., Goldbaum, M. H., Chuter, B., Girkin, C. A., et al. (2022) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2022.0244 · Cited by 53
Afrose, S., Song, W., Nemeroff, C. B., Lu, C., & Yao, D. D. (2022) · Communications Medicine · doi:10.1038/s43856-022-00165-w · Cited by 64
Machine Learning and Real-World Data to Predict Lung Cancer Risk in Routine Care
Chandran, U., Reps, J., Yang, R., Vachani, A., Maldonado, F., & Kalsekar, I. (2023) · Cancer Epidemiology, Biomarkers & Prevention · doi:10.1158/1055-9965.epi-22-0873 · Cited by 51
A Survey on Learning to Reject
Zhang, X.-Y., Xie, G.-S., Li, X., Mei, T., & Liu, C. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238024 · Cited by 88
The Use of Machine Learning for Image Analysis Artificial Intelligence in Clinical Microbiology
Burns, B. L., Rhoads, D. D., & Misra, A. (2023) · Journal of Clinical Microbiology · doi:10.1128/jcm.02336-21 · Cited by 67
Chemical Diversity and Classification of Secondary Metabolites in Nine Bryophyte Species
Peters, K., Treutler, H., Döll, S., Kindt, A. S. D., Hankemeier, T., & Neumann, S. (2019) · Metabolites · doi:10.3390/metabo9100222 · Cited by 60
Evaluating conservation dogs in the search for rare species
Bennett, E. M., Hauser, C. E., & Moore, J. L. (2019) · Conservation Biology · doi:10.1111/cobi.13431 · Cited by 52
Venkatesh, G., Grover, A., Srinivasaraghavan, G., & Rao, S. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa479 · Cited by 52
Predicting Firm-Level Bankruptcy in the Spanish Economy Using Extreme Gradient Boosting
Smith, M., & Álvarez, F. (2021) · Computational Economics · doi:10.1007/s10614-020-10078-2 · Cited by 57
Revealing protein-protein interactions at the transcriptome scale by sequencing
Johnson, K. L., Qi, Z., Yan, Z., Wen, X., Nguyen, T. C., Zaleta-Rivera, K., Chen, C.-J., Fan, X., Sriram, K., Wan, X., et al. (2021) · Molecular Cell · doi:10.1016/j.molcel.2021.07.006 · Cited by 52
Machine Learning Predictors of Extreme Events Occurring in Complex Dynamical Systems
Guth, S., & Sapsis, T. P. (2019) · Entropy · doi:10.3390/e21100925 · Cited by 51
Cruz, S. P., Guerra, R., Brázio, A., Cavaco, A. M., Antunes, M. D. C., & Passos, D. (2021) · Postharvest Biology and Technology · doi:10.1016/j.postharvbio.2021.111562 · Cited by 54
Choi, A., Choi, S. Y., Chung, K., Chung, H. S., Song, T., Choi, B., & Kim, J. H. (2023) · Scientific Reports · doi:10.1038/s41598-023-35617-3 · Cited by 47
Quasar, S. R., Sharma, R., Mittal, A., Sharma, M., Agarwal, D., & Díez, I. D. L. T. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17616-8 · Cited by 65
Lenters, V. C., Vermeulen, R. C. H., & Portengen, L. (2017) · Occupational and Environmental Medicine · doi:10.1136/oemed-2016-104231 · Cited by 51
Allele-specific binding of RNA-binding proteins reveals functional genetic variants in the RNA
Yang, E.-W., Bahn, J. H., Hsiao, Y.-H. E., Tan, B. X., Sun, Y., Fu, T., Zhou, B., Nostrand, E. L. V., Pratt, H., Freese, P., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-09292-w · Cited by 57
Genetic algorithms for feature selection when classifying severe chronic disorders of consciousness
Wutzl, B., Leibnitz, K., Rattay, F., Kronbichler, M., Murata, M., & Golaszewski, S. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0219683 · Cited by 54
Automatic detection of feeding- and drinking-related agonistic behavior and dominance in dairy cows
Foris, B., Thompson, A. J., von Keyserlingk, M. A. G., Melzer, N., & Weary, D. M. (2019) · Journal of Dairy Science · doi:10.3168/jds.2019-16697 · Cited by 51
Lo‐Ciganic, W., Huang, J. L., Zhang, H. H., Weiss, J. C., Kwoh, C. K., Donohue, J. M., Gordon, A. J., Cochran, G., Malone, D. C., Kuza, C. C., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0235981 · Cited by 51
U.S. Extreme Precipitation Weather Types Increased in Frequency During the 20th Century
Prein, A. F., & Mearns, L. O. (2021) · Journal of Geophysical Research Atmospheres · doi:10.1029/2020jd034287 · Cited by 57
Létinier, L., Jouganous, J., Benkebil, M., Bel‐Létoile, A., Goehrs, C., Singier, A., Rouby, F., Lacroix, C., Miremont, G., Micallef, J., et al. (2021) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.2266 · Cited by 66
The False Positive Problem of Automatic Bot Detection in Social Science ResearchPreprint
Rauchfleisch, A., & Kaiser, J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3565233 · Cited by 61
Ou, C., Chong, W., Duan, C., Zhang, X., Morgan, M. K., & Qian, Y. (2020) · European Radiology · doi:10.1007/s00330-020-07325-3 · Cited by 53
Importance-SMOTE: a synthetic minority oversampling method for noisy imbalanced data
Liu, J. (2021) · Soft Computing · doi:10.1007/s00500-021-06532-4 · Cited by 50
Performance comparison of TCR-pMHC prediction tools reveals a strong data dependency
Deng, L., Ly, C., Abdollahi, S., Zhao, Y., Prinz, I., & Bonn, S. (2023) · Frontiers in Immunology · doi:10.3389/fimmu.2023.1128326 · Cited by 50
MACE for Diagnosis of Dementia and MCI: Examining Cut-Offs and Predictive Values
Larner, A. J. (2019) · Diagnostics · doi:10.3390/diagnostics9020051 · Cited by 52
Xuan, P., Sun, H., Wang, X., Zhang, T., & Pan, S. (2019) · International Journal of Molecular Sciences · doi:10.3390/ijms20153648 · Cited by 53
Master Your Metrics with Calibration
Siblini, W., Fréry, J., He-Guelton, L., Oblé, F., & Wang, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-44584-3_36 · Cited by 71
Engel‐Haber, E., Zeilig, G., Haber, S., Worobey, L. A., & Kirshblum, S. (2020) · The Spine Journal · doi:10.1016/j.spinee.2020.05.551 · Cited by 53
Mpanya, D., Çelik, T., Klug, E., & Ntsinjana, H. N. (2021) · IJC Heart & Vasculature · doi:10.1016/j.ijcha.2021.100773 · Cited by 66
Machine learning models for prediction of adverse events after percutaneous coronary intervention
Niimi, N., Shiraishi, Y., Sawano, M., Ikemura, N., Inohara, T., Ueda, I., Fukuda, K., & Kohsaka, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-10346-1 · Cited by 52
A comparative study of different neural network models for landslide susceptibility mapping
Zhao, Z., He, Y., Yao, S., Wang, Y., Wang, W., Zhang, L., & Sun, Q. (2022) · Advances in Space Research · doi:10.1016/j.asr.2022.04.055 · Cited by 49
Predicting the risk of pipe failure using gradient boosted decision trees and weighted risk analysis
Barton, N. A., Hallett, S. H., Jude, S., & Tran, T. H. (2022) · npj Clean Water · doi:10.1038/s41545-022-00165-2 · Cited by 49
Machine Learning Models for Inpatient Glucose Prediction
Zale, A. D., & Mathioudakis, N. N. (2022) · Current Diabetes Reports · doi:10.1007/s11892-022-01477-w · Cited by 59
Assessing Predictability of Marine Heatwaves With Random Forests
Giamalaki, K., Beaulieu, C., & Prochaska, J. X. (2022) · Geophysical Research Letters · doi:10.1029/2022gl099069 · Cited by 54
Artificial intelligence and neoantigens: paving the path for precision cancer immunotherapy
Bulashevska, A., Nacsa, Z., Lang, F., Braun, M., Machyna, M., Diken, M., Childs, L. H., & König, R. (2024) · Frontiers in Immunology · doi:10.3389/fimmu.2024.1394003 · Cited by 56
Hybrid and Multifaceted Context-Aware Misbehavior Detection Model for Vehicular Ad Hoc Network
Ghaleb, F. A., Aizaini Maarof, M., Zainal, A., Al-Rimy, B. A. S., Saeed, F., & Al-Hadhrami, T. (2019) · IEEE Access · doi:10.1109/access.2019.2950805 · Cited by 49
Wozniakowska, P., & Eaton, D. W. (2020) · Geophysical Research Letters · doi:10.1029/2020gl089651 · Cited by 49
Detection of liver cirrhosis in standard T2-weighted MRI using deep transfer learning
Nowak, S., Mesropyan, N., Faron, A., Block, W., Reuter, M., Attenberger, U. I., Luetkens, J. A., & Sprinkart, A. M. (2021) · European Radiology · doi:10.1007/s00330-021-07858-1 · Cited by 52
Artificial Intelligence and Machine Learning: What You Always Wanted to Know but Were Afraid to Ask
Rattan, P., Penrice, D. D., & Simonetto, D. A. (2022) · Gastro Hep Advances · doi:10.1016/j.gastha.2021.11.001 · Cited by 60
Ren, T., Zhang, W., Li, S., Deng, L., Xue, C., Li, Z., Liu, S., Sun, J., & Zhou, J. (2022) · Diagnostic and Interventional Imaging · doi:10.1016/j.diii.2022.07.004 · Cited by 48
Metadata
- Authors
- Saito, T., & Rehmsmeier, M.
- Affiliations
- University of Bergen (NO)
- Corresponding author
- Saito, Takaya
- Rehmsmeier, Marc
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-03-04
- Volume
- 10
- Issue
- 3
- Pages
- e0118432
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4349800
- OpenAlex
- W1966716734
- Cited by
- 5,309
- References
- 56
- Citations by year
- 2026: 1482 · 2025: 762 · 2024: 528 · 2023: 505 · 2022: 551 · 2021: 563 · 2020: 402 · 2019: 283 · 2018: 133 · 2017: 64 · 2016: 27 · 2015: 3
- FWCI
- 74.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Universitetet i Bergen
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- class imbalance, binary classification, classifier evaluation, specificity, positive predictive value, sensitivity, bioinformatics classification, visual interpretability
- MeSH terms
- Classification, ROC Curve, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.