The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets
Open accessThe DOI matches its Crossref record; 55 of the 56 references it cites were found in a registry or an index.
Abstract
Binary classifiers are routinely evaluated with performance measures such as sensitivity and specificity, and performance is frequently illustrated with Receiver Operating Characteristics (ROC) plots. Alternative measures such as positive predictive value (PPV) and the associated Precision/Recall (PRC) plots are used less frequently. Many bioinformatics studies develop and evaluate classifiers that are to be applied to strongly imbalanced datasets in which the number of negatives outweighs the number of positives significantly. While ROC plots are visually appealing and provide an overview of a classifier's performance across a wide range of specificities, one can ask whether ROC plots could be misleading when applied in imbalanced classification scenarios. We show here that the visual interpretability of ROC plots in the context of imbalanced datasets can be deceptive with respect to conclusions about the reliability of classification performance, owing to an intuitive but wrong interpretation of specificity. PRC plots, on the other hand, can provide the viewer with an accurate prediction of future classification performance due to the fact that they evaluate the fraction of true positives among positive predictions. Our findings have potential implications for the interpretation of a large number of studies that use ROC plots on imbalanced datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“which is a more informative measure than ROC analysis in imbalanced scenarios59.”
From K-STAMM: a knowledge-enhanced spatial - temporal attention model with multimodal fusion for pneumonia prediction (2026) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Because extreme prediction is by definition an imbalanced classification problem, we base our analysis on precision-recall plots which are well-suited for this case [42].”
From Learning skillful medium-range global weather forecasting (2023) · cites this paper as [42]
In the citing paper
“Furthermore, careful consideration should be given when selecting evaluation metrics, as the precision-recall curve is more informative for imbalanced datasets than the most commonly used receiver-operating characteristic curve (true positive rate vs. false positive rate) (Fig. 5C).115”
From Use of artificial intelligence for liver diseases: A survey from the EASL congress 2024 (2024) · cites this paper as [115]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Bartel (2004) · doi:10.1016/s0092-8674(04)00045-5 · Cited by 34,714
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 3×
“Here, we review the theoretical background of commonly used evaluation measures, specifically ROC [24, 25], PRC [26], concentrated ROC (CROC) [27], and Cost Curves (CC) [28].”
BackgroundMethodThe meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
Bioconductor: open software development for computational biology and bioinformatics
Gentleman, R. C., Carey, V. J., Bates, D. M., Bolstad, B., Dettling, M., Dudoit, S., Ellis, B., Gautier, L., Ge, Y., Gentry, J., et al. (2004) · Genome Biology · doi:10.1186/gb-2004-5-10-r80 · Cited by 12,172 · cited 1×
“Several integrated machine learning and bioinformatics platforms such as WEKA [34] and Bioconductor [42, 43] also have basic functions or libraries for making ROC and PRC plots.”
MethodA training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
R: A Language for Data Analysis and Graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,220
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Cited by
Items 301–400 of 1,023
Lopes, R. R., Bleijendaal, H., Ramos, L. A., Verstraelen, T. E., Amin, A. S., Wilde, A. A. M., Pinto, Y. M., de Mol, B. A. J. M., & Marquering, H. A. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104262 · Cited by 85
Robinson, M. C., Glen, R. C., & Lee, A. A. (2020) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-019-00274-0 · Cited by 74
Adachi, T., Nakamura, M., Shintani, T., Mitsuyoshi, T., Kakino, R., Ogata, T., Ono, T., Tanabe, H., Kokubo, M., Sakamoto, T., et al. (2021) · Medical Physics · doi:10.1002/mp.14769 · Cited by 81
Hằng, H. T., Mallick, J., Alqadhi, S. D., Bindajam, A. A., & Abdo, H. G. (2024) · Environmental Technology & Innovation · doi:10.1016/j.eti.2024.103655 · Cited by 77
Luo, J., Deng, M., Zhang, X., & Sun, X. (2023) · Genome Research · doi:10.1101/gr.278001.123 · Cited by 92
Peeking into a black box, the fairness and generalizability of a MIMIC-III benchmarking model.
Röösli, E., Bozkurt, S., & Hernandez-Boussard, T. (2022) · Scientific Data · doi:10.1038/s41597-021-01110-7 · Cited by 73
Automating the search for a patent’s prior art with a full text similarity search
Helmers, L., Horn, F., Biegler, F., Oppermann, T., & Müller, K. (2019) · PLoS ONE · doi:10.1371/journal.pone.0212103 · Cited by 91
Breath-based non-invasive diagnosis of Alzheimer’s disease: a pilot study
Tiele, A., Wicaksono, A. N., Daulton, E., Ifeachor, E., Eyre, V., Clarke, S., Timings, L., Pearson, S., Covington, J. A., & Li, X. (2019) · Journal of Breath Research · doi:10.1088/1752-7163/ab6016 · Cited by 73
Prendin, F., Del Favero, S., Vettoretti, M., Sparacino, G., & Facchinetti, A. (2021) · Sensors · doi:10.3390/s21051647 · Cited by 72
Jas, K., Mangalathu, S., & Dodagoudar, G. R. (2024) · Computers and Geotechnics · doi:10.1016/j.compgeo.2023.106051 · Cited by 78
A Monte Carlo-based framework for risk-return analysis in mineral prospectivity mapping
Wang, Z., Yin, Z., Caers, J., & Zuo, R. (2020) · Geoscience Frontiers · doi:10.1016/j.gsf.2020.02.010 · Cited by 76
Evaluating Polygenic Risk Scores for Breast Cancer in Women of African Ancestry
Du, Z., Gao, G., Adedokun, B., Ahearn, T. U., Lunetta, K. L., Zirpoli, G., Troester, M. A., Ruiz-Narváez, E., Haddad, S. A., PalChoudhury, P., et al. (2021) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djab050 · Cited by 73
Standardized Reporting of Machine Learning Applications in Urology: The STREAM-URO Framework
Kwong, J. C. C., McLoughlin, L. C., Haider, M. A., Goldenberg, M. G., Erdman, L., Rickard, M., Lorenzo, A. J., Hung, A. J., Farcas, M. A., Goldenberg, L., et al. (2021) · European Urology Focus · doi:10.1016/j.euf.2021.07.004 · Cited by 78
DeMars, M. M., & Perruso, C. (2022) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2022.1283 · Cited by 97
Choudhury, A., Mondal, A., & Sarkar, S. (2024) · The European Physical Journal Special Topics · doi:10.1140/epjs/s11734-024-01308-x · Cited by 104
Mathioudakis, N. N., Abusamaan, M. S., Shakarchi, A. F., Sokolinsky, S., Fayzullin, S., McGready, J., Zilbermint, M., Saria, S., & Golden, S. H. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.30913 · Cited by 67
Prudent, X., Parra, G., Schwede, P., Roscito, J. G., & Hiller, M. (2016) · Molecular Biology and Evolution · doi:10.1093/molbev/msw098 · Cited by 71
“Given that the vast majority of functional elements are negatives, receiver operating characteristics plots that compare sensitivity and specificity are not suitable on such highly imbalanced datasets (Saito and Rehmsmeier 2015).”
BackgroundInfluentialDynamic Prediction in Clinical Survival Analysis Using Temporal Convolutional Networks
Jarrett, D., Yoon, J., & van der Schaar, M. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2019.2929264 · Cited by 95
Predicting bankruptcy of local government: A machine learning approach
Antulov-Fantulin, N., Lagravinese, R., & Resce, G. (2021) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2021.01.014 · Cited by 90
Computer vision-based citrus tree detection in a cultivated environment using UAV imagery
Dönmez, C., VİLLİ, O., Berberoğlu, S., & Çilek, A. (2021) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2021.106273 · Cited by 78
Liu, Y., Ko, C. Y., Hall, B. L., & Cohen, M. E. (2023) · Journal of the American College of Surgeons · doi:10.1097/xcs.0000000000000556 · Cited by 81
Liu, D., Zhang, J., Qi, Y., Wu, Y., & Zhang, Y. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3381774 · Cited by 128
High-Throughput Deep Learning Detection of Mitral Regurgitation
Vrudhula, A., Duffy, G., Vukadinovic, M., Liang, D. H., Cheng, S. C., & Ouyang, D. (2024) · Circulation · doi:10.1161/circulationaha.124.069047 · Cited by 82
Lötsch, J., Schiffmann, S., Schmitz, K., Brunkhorst, R., Lerch, F., Ferreirós, N., Wicker, S., Tegeder, I., Geißlinger, G., & Ultsch, A. (2018) · Scientific Reports · doi:10.1038/s41598-018-33077-8 · Cited by 79
Natural language processing for automated detection of incidental durotomy
Karhade, A. V., Bongers, M. E. R., Groot, O. Q., Kazarian, E. R., Cha, T. D., Fogel, H. A., Hershman, S. H., Tobert, D. G., Schoenfeld, A. J., Bono, C. M., et al. (2019) · The Spine Journal · doi:10.1016/j.spinee.2019.12.006 · Cited by 76
A Novel Performance Measure for Machine Learning Classification
Gong, M. (2021) · International Journal of Managing Information Technology · doi:10.5121/ijmit.2021.13101 · Cited by 90
Drug repositioning with adaptive graph convolutional networks
Sun, X., Xiao, J., Lu, Z., Tang, J. P., & Li, M. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad748 · Cited by 78
Yang, J., Li, A., Li, Y., Guo, X., & Wang, M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty848 · Cited by 80
Pandey, G. S., Pandey, O. P., Rogers, A. J., Ahsen, M. E., Hoffman, G. E., Raby, B. A., Weiss, S. T., Schadt, E. E., & Bunyavanich, S. (2018) · Scientific Reports · doi:10.1038/s41598-018-27189-4 · Cited by 76
Santana, M. V. S., & Silva-Jr, F. P. (2021) · BMC Chemistry · doi:10.1186/s13065-021-00737-2 · Cited by 68
Cho, W. K., Lee, Y. J., Joo, H. A., Jeong, I. S., Choi, Y., Nam, S. Y., Kim, S. Y., & Choi, S. (2021) · The Laryngoscope · doi:10.1002/lary.29595 · Cited by 68
Chandrasekar, V., Ansari, M. Y., Singh, A. V., Shahab Uddin, M. S., Prabhu, K. S., Dash, S., Khodor, S. A., Terranegra, A., Avella, M., & Dakua, S. P. (2023) · IEEE Access · doi:10.1109/access.2023.3272987 · Cited by 74
Online suicide prevention through optimised text classification
Desmet, B., & Hoste, V. (2018) · Information Sciences · doi:10.1016/j.ins.2018.02.014 · Cited by 66
A systematic exploration of satellite radar coherence methods for rapid landslide detection
Burrows, K., Walters, R. J., Milledge, D. G., & Densmore, A. L. (2020) · Natural hazards and earth system sciences · doi:10.5194/nhess-20-3197-2020 · Cited by 82
Survival prediction of patients with sepsis from age, sex, and septic episode number alone
Chicco, D., & Jurman, G. (2020) · Scientific Reports · doi:10.1038/s41598-020-73558-3 · Cited by 79
Felsche, E., & Ludwig, R. (2021) · Natural hazards and earth system sciences · doi:10.5194/nhess-21-3679-2021 · Cited by 73
Kidney Tumor Semantic Segmentation Using Deep Learning: A Survey of State-of-the-Art
Abdelrahman, A., & Viriri, S. (2022) · Journal of Imaging · doi:10.3390/jimaging8030055 · Cited by 81
Choi, M. H., Kim, D., Choi, E. J., Jung, Y. J., Choi, Y. J., Cho, J. H., & Jeong, S. H. (2022) · Scientific Reports · doi:10.1038/s41598-022-11226-4 · Cited by 85
Yuan, Q., Chen, S., Wang, Y., Zhao, H., & Yang, Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac444 · Cited by 68
Data-Driven Anomaly Recognition for Unsupervised Model-Free Fault Detection in Artificial Pancreas
Meneghetti, L., Terzi, M., Favero, S. D., Susto, G. A., & Cobelli, C. (2018) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2018.2885963 · Cited by 74
Analyzing Factors Associated with Fatal Road Crashes: A Machine Learning Approach
Ghandour, A. J., Hammoud, H., & Al‐Hajj, S. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17114111 · Cited by 97
Mufti, H. N., Hirsch, G. M., Abidi, S., & Abidi, S. S. R. (2019) · JMIR Medical Informatics · doi:10.2196/14993 · Cited by 75
Miché, M., Studerus, E., Meyer, A. H., Gloster, A. T., Beesdo‐Baum, K., Wïttchen, H., & Lieb, R. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.11.093 · Cited by 76
Heterogeneous graph attention network based on meta-paths for lncRNA–disease association prediction
Zhao, X., Zhao, X., & Yin, M. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab407 · Cited by 71
Explainable Boosting Machines for Slope Failure Spatial Predictive Modeling
Maxwell, A. E., Sharma, M., & Donaldson, K. A. (2021) · Remote Sensing · doi:10.3390/rs13244991 · Cited by 85
Dhok, S. G., Pimpalkhute, V., Chandurkar, A., Bhurane, A. A., Sharma, M. D., & Acharya, U. R. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103691 · Cited by 66
Revealing cytotoxic substructures in molecules using deep learning
Webel, H. E., Kimber, T. B., Radetzki, S., Neuenschwander, M., Nazaré, M., & Volkamer, A. (2020) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-020-00310-4 · Cited by 66
Personalized prediction of early childhood asthma persistence: A machine learning approach
Bose, S., Kenyon, C. C., & Masino, A. J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0247784 · Cited by 75
Transparent Exploration of Machine Learning for Biomarker Discovery from Proteomics and Omics Data
Torun, F. M., Winter, S. V., Doll, S., Riese, F. M., Vorobyev, A., Müller, J., Geyer, P. E., & Strauss, M. T. (2022) · Journal of Proteome Research · doi:10.1021/acs.jproteome.2c00473 · Cited by 83
Zaghloul, M., Barakat, S., & Rezk, A. (2024) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2024.103865 · Cited by 110
Amebleh, J., Igba, E., & Ijiga, O. M. (2021) · International Journal of Scientific Research in Science Engineering and Technology · doi:10.32628/ijsrset214418 · Cited by 76
Brinker, T. J., Kiehl, L., Schmitt, M., Jutzi, T. B., Krieghoff-Henning, E. I., Krahl, D., Kutzner, H., Gholam, P., Haferkamp, S., Klode, J., et al. (2021) · European Journal of Cancer · doi:10.1016/j.ejca.2021.05.026 · Cited by 65
Liu, Y.-X., Liu, X., Cen, C., Li, X., Liu, J., Ming, Z., Yu, S., Tang, X., Zhou, L., Yu, J., et al. (2021) · Hepatobiliary & pancreatic diseases international · doi:10.1016/j.hbpd.2021.08.004 · Cited by 68
Hong, W., Zhou, X., Jin, S., Lu, Y., Pan, J., Lin, Q., Yang, S., Xu, T., Basharat, Z., Zippi, M., et al. (2022) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2022.819267 · Cited by 81
Tripathi, P., & Pandey, P. N. (2017) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2017.04.027 · Cited by 65
Zhang, X., Walsh, R., Whiffin, N., Buchan, R. J., Midwinter, W., Wilk, A., Govind, R., Li, N., Ahmad, M., Mazzarotto, F., et al. (2020) · Genetics in Medicine · doi:10.1038/s41436-020-00972-3 · Cited by 73
Li, G., Liu, Q., Ren, W., Qiao, W., Ma, B., & Wan, J. (2020) · Measurement · doi:10.1016/j.measurement.2020.108693 · Cited by 70
Zhou, Q. M., Zhe, L., Brooke, R. J., Hudson, M. M., & Yuan, Y. (2021) · Diagnostic and Prognostic Research · doi:10.1186/s41512-021-00102-w · Cited by 73
When Do Politicians Use Populist Rhetoric? Populism as a Campaign Gamble
Dai, Y., & Kustov, A. (2022) · Political Communication · doi:10.1080/10584609.2022.2025505 · Cited by 94
Transformers for Molecular Property Prediction: Lessons Learned from the Past Five Years
Sultan, A., Sieg, J., Mathea, M., & Volkamer, A. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c00747 · Cited by 91
Cardinal, M. R., Destrempes, F., Soulez, G., & Cloutier, G. (2018) · IEEE Transactions on Ultrasonics Ferroelectrics and Frequency Control · doi:10.1109/tuffc.2018.2851846 · Cited by 69
Chen, Y., Zhang, G., Ma, Y., Kang, J. U., & Kwan, C. (2021) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2020.3047524 · Cited by 63
Gene regulatory network inference in single-cell biology
Akers, K., & Murali, T. M. (2021) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2021.04.007 · Cited by 77
Udriștoiu, A. L., Cazacu, I. M., Gruionu, L. G., Gruionu, G., Iacob, A. V., Burtea, D. E., Ungureanu, B. S., Costache, M. I., Constantin, A., Popescu, C. F., et al. (2021) · PLOS ONE · doi:10.1371/journal.pone.0251701 · Cited by 63
“We used the Precision-Recall as another metrics of the model’s prediction [16].”
MethodMovahedi, F., Padman, R., & Antaki, J. F. (2021) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2021.07.041 · Cited by 93
Cyberbullying Detection on Social Media Using Stacking Ensemble Learning and Enhanced BERT
Muneer, A., Alwadain, A. S., Ragab, M. G., & Alqushaibi, A. (2023) · Information · doi:10.3390/info14080467 · Cited by 89
Xuan, P., Gao, L., Sheng, N., Zhang, T., & Nakaguchi, T. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3039502 · Cited by 62
Kareemi, H., Vaillancourt, C., Rosenberg, H., Fournier, K., & Yadav, K. (2021) · Academic Emergency Medicine · doi:10.1111/acem.14190 · Cited by 72
Ilan, Y. (2020) · Frontiers in Digital Health · doi:10.3389/fdgth.2020.569178 · Cited by 73
Sheng, N., Wang, Y., Huang, L., Gao, L., Cao, Y., Xie, X., & Fu, Y. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad276 · Cited by 66
Selecting textual analysis tools to classify sustainability information in corporate reporting
Maibaum, F., Kriebel, J., & Foege, J. N. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114269 · Cited by 88
Ren, Z., Zhu, J., Gao, Y., Yin, Q., Hu, M., Dai, L., Deng, C., Yi, L., Deng, K., Wang, Y., et al. (2018) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2018.02.181 · Cited by 66
From Feature Engineering and Topics Models to Enhanced Prediction Rates in Phishing Detection
Gualberto, É. S., de Sousa, R. T., Vieira, T. P. D. B., da Costa, J. P. C. L., & Duque, C. G. (2020) · IEEE Access · doi:10.1109/access.2020.2989126 · Cited by 90
Flood Detection Using Multi-Modal and Multi-Temporal Images: A Comparative Study
Islam, K. A., Uddin, M. S., Kwan, C., & Li, J. (2020) · Remote Sensing · doi:10.3390/rs12152455 · Cited by 65
On machine learning effectiveness for malware detection in Android OS using static analysis data
Syrris, V., & Geneiatakis, D. (2021) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2021.102794 · Cited by 74
Lauritsen, S. M., Thiesson, B., Jørgensen, M. J., Riis, A. H., Espelund, U. S., Weile, J. B., & Lange, J. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00529-x · Cited by 61
Twelve key challenges in medical machine learning and solutions
Ellis, R. J., Sander, R. M., & Limon, A. (2022) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2022.100068 · Cited by 61
Bai, Z., Liu, Q., & Liu, Y. (2022) · Natural Resources Research · doi:10.1007/s11053-022-10100-4 · Cited by 69
Damage detection with ultrasonic guided waves using machine learning and aggregated baselines
Nerlikar, V., Mesnil, O., Miorelli, R., & D’Almeida, O. (2023) · Structural Health Monitoring · doi:10.1177/14759217231169719 · Cited by 68
Lee, M. S., Shin, T. G., Lee, Y. J., Kim, D. H., Choi, S., Cho, H., Lee, M.-J., Jeong, K. Y., Kim, W. Y., Min, Y. G., et al. (2025) · European Heart Journal · doi:10.1093/eurheartj/ehaf004 · Cited by 83
CREST: Cross-Reference to Exchange-based Stock Trend Prediction using Long Short-Term Memory
Thakkar, A., & Chaudhari, K. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.03.328 · Cited by 63
Deep learning to map concentrated animal feeding operations
Handan‐Nader, C., & Ho, D. E. (2019) · Nature Sustainability · doi:10.1038/s41893-019-0246-x · Cited by 60
Sheng, N., Cui, H., Zhang, T., & Xuan, P. (2020) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa067 · Cited by 61
A Crash Injury Model Involving Autonomous Vehicle: Investigating of Crash and Disengagement Reports
Sinha, A., Vu, V., Chand, S., Wijayaratna, K. P., & Dixit, V. V. (2021) · Sustainability · doi:10.3390/su13147938 · Cited by 73
Giglioni, V., García‐Macías, E., Venanzi, I., Ierimonti, L., & Ubertini, F. (2021) · Engineering Structures · doi:10.1016/j.engstruct.2021.113029 · Cited by 66
Lo‐Ciganic, W., Donohue, J. M., Yang, Q., Huang, J. L., Chang, C., Weiss, J. C., Guo, J. S., Zhang, H. H., Cochran, G., Gordon, A. J., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00062-0 · Cited by 66
A Comparison of Techniques for Class Imbalance in Deep Learning Classification of Breast Cancer
Walsh, R., & Tardy, M. (2022) · Diagnostics · doi:10.3390/diagnostics13010067 · Cited by 79
Shickel, B., Loftus, T. J., Ruppert, M. M., Upchurch, G. R., Ozrazgat‐Baslanti, T., Rashidi, P., & Bihorac, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-27418-5 · Cited by 83
A Survey of Methods for Addressing Imbalance Data Problems in Agriculture Applications
Miftahushudur, T., Sahin, H. M., Grieve, B., & Yin, H. (2025) · Remote Sensing · doi:10.3390/rs17030454 · Cited by 91
Spatiotemporal Bayesian networks for malaria prediction
Haddawy, P., Hasan, A. H. M. I., Kasantikul, R., Lawpoolsri, S., Sa-angchai, P., Kaewkungwal, J., & Singhasivanon, P. (2017) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2017.12.002 · Cited by 60
Long, N. P., Yoon, S. J., Anh, N. H., Nghi, T. D., Lim, D. K., Hong, Y. J., Hong, S., & Kwon, S. W. (2018) · Metabolomics · doi:10.1007/s11306-018-1404-2 · Cited by 66
TriPepSVM: de novo prediction of RNA-binding proteins based on short amino acid motifs
Bressin, A., Schulte-Sasse, R., Figini, D., Urdaneta, E. C., Beckmann, B. M., & Marsico, A. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz203 · Cited by 60
Majzoub, H. A., Elgedawy, I., Akaydın, Ö., & Ulukök, M. K. (2020) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-019-04336-1 · Cited by 73
Computational models for predicting drug responses in cancer research
Azuaje, F. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw065 · Cited by 59
Dwivedi, A. K., & Shukla, R. (2019) · Cancer Reports · doi:10.1002/cnr2.1211 · Cited by 74
Doyle, O., Leavitt, N., & Rigg, J. (2020) · Scientific Reports · doi:10.1038/s41598-020-67013-6 · Cited by 70
Machine learning models predicting multidrug resistant urinary tract infections using “DsaaS”
Mancini, A., Vito, L., Marcelli, E., Piangerelli, M., Leone, R. D., Pucciarelli, S., & Merelli, E. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-03566-7 · Cited by 59
Pipe breaks and estimating the impact of pressure control in water supply networks
Jara‐Arriagada, C., & Stoianov, I. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.107525 · Cited by 68
Anomaly Detection on Data Streams for Smart Agriculture
Moso, J. C., Cormier, S., de Runz, C., Fouchal, H., & Wandeto, J. (2021) · Agriculture · doi:10.3390/agriculture11111083 · Cited by 65
Chungath, T. T., Nambiar, A., & Mittal, A. (2023) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2022.3221127 · Cited by 70
Metadata
- Authors
- Saito, T., & Rehmsmeier, M.
- Affiliations
- University of Bergen (NO)
- Corresponding author
- Saito, Takaya
- Rehmsmeier, Marc
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-03-04
- Volume
- 10
- Issue
- 3
- Pages
- e0118432
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4349800
- OpenAlex
- W1966716734
- Cited by
- 5,309
- References
- 56
- Citations by year
- 2026: 1482 · 2025: 762 · 2024: 528 · 2023: 505 · 2022: 551 · 2021: 563 · 2020: 402 · 2019: 283 · 2018: 133 · 2017: 64 · 2016: 27 · 2015: 3
- FWCI
- 74.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Universitetet i Bergen
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- class imbalance, binary classification, classifier evaluation, specificity, positive predictive value, sensitivity, bioinformatics classification, visual interpretability
- MeSH terms
- Classification, ROC Curve, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.