The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets
Open accessThe DOI matches its Crossref record; 55 of the 56 references it cites were found in a registry or an index.
Abstract
Binary classifiers are routinely evaluated with performance measures such as sensitivity and specificity, and performance is frequently illustrated with Receiver Operating Characteristics (ROC) plots. Alternative measures such as positive predictive value (PPV) and the associated Precision/Recall (PRC) plots are used less frequently. Many bioinformatics studies develop and evaluate classifiers that are to be applied to strongly imbalanced datasets in which the number of negatives outweighs the number of positives significantly. While ROC plots are visually appealing and provide an overview of a classifier's performance across a wide range of specificities, one can ask whether ROC plots could be misleading when applied in imbalanced classification scenarios. We show here that the visual interpretability of ROC plots in the context of imbalanced datasets can be deceptive with respect to conclusions about the reliability of classification performance, owing to an intuitive but wrong interpretation of specificity. PRC plots, on the other hand, can provide the viewer with an accurate prediction of future classification performance due to the fact that they evaluate the fraction of true positives among positive predictions. Our findings have potential implications for the interpretation of a large number of studies that use ROC plots on imbalanced datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“which is a more informative measure than ROC analysis in imbalanced scenarios59.”
From K-STAMM: a knowledge-enhanced spatial - temporal attention model with multimodal fusion for pneumonia prediction (2026) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Because extreme prediction is by definition an imbalanced classification problem, we base our analysis on precision-recall plots which are well-suited for this case [42].”
From Learning skillful medium-range global weather forecasting (2023) · cites this paper as [42]
In the citing paper
“Furthermore, careful consideration should be given when selecting evaluation metrics, as the precision-recall curve is more informative for imbalanced datasets than the most commonly used receiver-operating characteristic curve (true positive rate vs. false positive rate) (Fig. 5C).115”
From Use of artificial intelligence for liver diseases: A survey from the EASL congress 2024 (2024) · cites this paper as [115]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Bartel (2004) · doi:10.1016/s0092-8674(04)00045-5 · Cited by 34,714
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 3×
“Here, we review the theoretical background of commonly used evaluation measures, specifically ROC [24, 25], PRC [26], concentrated ROC (CROC) [27], and Cost Curves (CC) [28].”
BackgroundMethodThe meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
Bioconductor: open software development for computational biology and bioinformatics
Gentleman, R. C., Carey, V. J., Bates, D. M., Bolstad, B., Dettling, M., Dudoit, S., Ellis, B., Gautier, L., Ge, Y., Gentry, J., et al. (2004) · Genome Biology · doi:10.1186/gb-2004-5-10-r80 · Cited by 12,172 · cited 1×
“Several integrated machine learning and bioinformatics platforms such as WEKA [34] and Bioconductor [42, 43] also have basic functions or libraries for making ROC and PRC plots.”
MethodA training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
R: A Language for Data Analysis and Graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,220
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Cited by
Items 501–600 of 1,023
Machine Learning Algorithms for understanding the determinants of under-five Mortality
Saroj, R. K., Yadav, P. K., Singh, R. K., & Chilyabanyama, O. N. (2022) · BioData Mining · doi:10.1186/s13040-022-00308-8 · Cited by 57
What Could Go Wrong? Introspective Radar Odometry in Challenging Environments
Aldera, R., Martini, D. D., Gadd, M., & Newman, P. M. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8917111 · Cited by 48
Local warming and violent armed conflict in Africa
van Weezel, S. (2020) · World Development · doi:10.1016/j.worlddev.2019.104708 · Cited by 47
Multi-Branching Temporal Convolutional Network for Sepsis Prediction
Wang, Z., & Yao, B. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3092835 · Cited by 55
Lu, Z., Sim, J.-A., Wang, J. X., Forrest, C. B., Krull, K. R., Srivastava, D. K., Hudson, M. M., Robison, L. L., Baker, J. N., & Huang, I. (2021) · Journal of Medical Internet Research · doi:10.2196/26777 · Cited by 52
Gleason, A. C., Ghadge, G. D., Chen, J., Sonobe, Y., & Roos, R. P. (2022) · PLoS ONE · doi:10.1371/journal.pone.0256411 · Cited by 54
Evaluating Eye Movement Event Detection: A Review of the State of the Art
Startsev, M., & Zemblys, R. (2022) · Behavior Research Methods · doi:10.3758/s13428-021-01763-7 · Cited by 56
McGibbon, M., Money-Kyrle, S., Blay, V., & Houston, D. R. (2022) · Journal of Advanced Research · doi:10.1016/j.jare.2022.07.001 · Cited by 49
Nasal DNA methylation at three CpG sites predicts childhood allergic disease
van Breugel, M., Qi, C., Xu, Z., Pedersen, C. T., Petoukhov, I., Vonk, J. M., Gehring, U., Berg, M., Bügel, M., Carpaij, O. A., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-35088-6 · Cited by 52
Machine Learning in Clinical Trials: A Primer with Applications to Neurology
Miller, M. I., Shih, L. C., & Kolachalama, V. B. (2023) · Neurotherapeutics · doi:10.1007/s13311-023-01384-2 · Cited by 67
Kazmierski, M., Welch, M. L., Kim, S., McIntosh, C., Rey‐McIntyre, K., Huang, S. H., Patel, T., Tadic, T., Milosevic, M., Liu, F., et al. (2023) · Cancer Research Communications · doi:10.1158/2767-9764.crc-22-0152 · Cited by 59
Ghislat, G., Hernández-Hernández, S., Piyawajanusorn, C., & Ballester, P. J. (2024) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2024.2403639 · Cited by 61
A diagnostic germ cell score for immature testicular tissue at risk of germ cell loss
Heckmann, L., Langenstroth, D., Pock, T., Wistuba, J., Stukenborg, J., Zitzmann, M., Kliesch, S., Schlatt, S., & Neuhaus, N. (2018) · Human Reproduction · doi:10.1093/humrep/dey025 · Cited by 49
Deep Neural Architectures for Highly Imbalanced Data in Bioinformatics
Bugnon, L. A., Yones, C., Milone, D. H., & Stegmayer, G. S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2914471 · Cited by 47
Sun, X., Zhang, X., Muir, D. C. G., & Zeng, E. Y. (2020) · Environmental Science & Technology · doi:10.1021/acs.est.0c01437 · Cited by 48
Barata, F., Tinschert, P., Rassouli, F., Steurer-Stey, C. A., Fleisch, E., Puhan, M. A., Brutsche, M. H., Kotz, D. F., & Kowatsch, T. (2020) · Journal of Medical Internet Research · doi:10.2196/18082 · Cited by 51
Chang, J., Binuesa, F., Canêo, L. F., Turquetto, A. L. R., Arita, E. C. T. C., Barbosa, A. C., Fernandes, A. M. D. S., Trindade, E. M., Jatene, F. B., Dossou, P.-E., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0238199 · Cited by 52
Evaluating predictive models in reproductive medicine
Curchoe, C. L., Farias, A. F.-S., Mendizabal‐Ruiz, G., & Chávez-Badiola, A. (2020) · Fertility and Sterility · doi:10.1016/j.fertnstert.2020.09.159 · Cited by 55
Kawakita, S., Beaumont, J. L., Jucaud, V., & Everly, M. J. (2020) · Scientific Reports · doi:10.1038/s41598-020-75473-z · Cited by 50
Controls of outbursts of moraine-dammed lakes in the greater Himalayan region
Fischer, M., Korup, O., Veh, G., & Walz, A. (2021) · The Cryosphere · doi:10.5194/tc-15-4145-2021 · Cited by 50
Benchmark Evaluation of Protein–Protein Interaction Prediction Algorithms
Dunham, B., & Ganapathiraju, M. K. (2021) · Molecules · doi:10.3390/molecules27010041 · Cited by 47
Lessons and tips for designing a machine learning study using EHR data
Arbet, J., Brokamp, C., Meinzen-Derr, J., Trinkley, K. E., & Spratt, H. M. (2021) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.513 · Cited by 46
“when the number of subjects per class is not approximately equal), such as over-/undersampling, cost-sensitive methods, or precision and recall curves [39,40].”
MethodVidal-Ribas, P., Janiri, D., Doucet, G. E., Pornpattananangkul, N., Nielson, D. M., Frangou, S., & Stringaris, A. (2021) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2020.20020120 · Cited by 45
Automatic pixel-wise detection of evolving cracks on rock surface in video data
Ai, D., Jiang, G., Lam, S.-K., He, P., & Li, C. (2020) · Automation in Construction · doi:10.1016/j.autcon.2020.103378 · Cited by 46
Variationally regularized graph-based representation learning for electronic health records
Zhu, W., & Razavian, N. (2021) · ACM Conference on Health, Inference, and Learning (CHIL) · doi:10.1145/3450439.3451855 · Cited by 63
Perez‐Valero, E., López-Gordo, M. A., Morillas, C. A., Pelayo, F. J., & Vaquero‐Blasco, M. A. (2021) · Journal of Alzheimer s Disease · doi:10.3233/jad-201455 · Cited by 48
How to identify early defaults in online lending: A cost-sensitive multi-layer learning framework
Li, Z., Zhang, J., Yao, X., & Kou, G. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106963 · Cited by 52
An Analysis of Performance Metrics for Imbalanced Classification
Gaudreault, J.-G., Branco, P., & da Gama, J. M. P. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88942-5_6 · Cited by 64
Azhar, A., Ariff, N. M., Bakar, M. A. A., & Roslan, A. (2022) · Sustainability · doi:10.3390/su14074101 · Cited by 46
Anomaly Detection in Financial Time Series by Principal Component Analysis and Neural Networks
Crépey, S., Lehdili, N., Madhar, N., & Thomas, M. (2022) · Algorithms · doi:10.3390/a15100385 · Cited by 66
Electrocardiogram Detection of Pulmonary Hypertension Using Deep Learning
ARAS, M. A., ABREAU, S., MILLS, H., RADHAKRISHNAN, L., KLEIN, L., MANTRI, N., RUBIN, B., BARRIOS, J., CHEHOUD, C., KOGAN, E., et al. (2023) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2022.12.016 · Cited by 46
Assessment of community efforts to advance network-based prediction of protein–protein interactions
Wang, X., Madeddu, L., Spirohn, K., Martini, L., Fazzone, A., Becchetti, L., Wytock, T. P., Kovács, I. A., Balogh, O. M., Benczik, B., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-37079-7 · Cited by 58
Kilpatrick, K. A., Podestá, G., Williams, E. D., Walsh, S. M., & Minnett, P. J. (2019) · Journal of Atmospheric and Oceanic Technology · doi:10.1175/jtech-d-18-0103.1 · Cited by 52
Yñiguez, A. T., & Ottong, Z. J. (2019) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2019.136173 · Cited by 51
Sampedro-Gómez, J., Dorado-Díaz, P. I., Vicente-Palacios, V., Sánchez-Puente, A., Jiménez-Navarro, M. F., Román, J. A. S., Galindo‐Villardón, P., Sanchez, P. L., & Fernández‐Avilés, F. (2020) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2020.01.027 · Cited by 51
Maxwell, A. E., Sharma, M., Kite, J. S., Donaldson, K. A., Thompson, J. A., Bell, M., & Maynard, S. M. (2020) · Remote Sensing · doi:10.3390/rs12030486 · Cited by 49
AI-based analysis of CT images for rapid triage of COVID-19 patients
Xu, Q., Zhan, X., Zhou, Z., Li, Y., Xie, P., Zhang, S., Li, X., Yu, Y., Zhou, C., Zhang, L., et al. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00446-z · Cited by 46
Incremental learning strategies for credit cards fraud detection
Lebichot, B., Paldino, G. M., Siblini, W., He-Guelton, L., Oblé, F., & Bontempi, G. (2021) · International Journal of Data Science and Analytics · doi:10.1007/s41060-021-00258-0 · Cited by 50
BiGAN: LncRNA-disease association prediction based on bidirectional generative adversarial network
Yang, Q., & Li, X. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04273-7 · Cited by 45
Chaves, J. M. Z., Wentland, A. L., Desai, A., Banerjee, I., Kaur, G., Correa, R., Boutin, R. D., Maron, D. J., Rodríguez, F., Sandhu, A. T., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-47895-y · Cited by 57
Phan, P., Budhram, B., Zhang, Q., Rivers, C. S., Noonan, V. K., Plashkes, T., Wai, E. K., Paquet, J., Roffey, D. M., Tsai, E. C., et al. (2018) · The Spine Journal · doi:10.1016/j.spinee.2018.08.016 · Cited by 55
Level-2 node clustering coefficient-based link prediction
Kumar, A., Singh, S. S., Singh, K., & Biswas, B. (2019) · Applied Intelligence · doi:10.1007/s10489-019-01413-8 · Cited by 45
Ten quick tips for deep learning in biology
Lee, B. D., Gitter, A., Greene, C. S., Raschka, S., Maguire, F., Titus, A., Kessler, M. D., Lee, A. J., Chevrette, M. G., Stewart, P. A., et al. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009803 · Cited by 51
Ší́p, V., Hashemi, M., Vattikonda, A. N., Woodman, M., Wang, H., Makhalova, J., Villalon, S. M., Guye, M., Bartoloméi, F., & Jirsa, V. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008689 · Cited by 48
Fenn, A., Davis, C., Buckland, D. M., Kapadia, N. N., Nichols, M., Gao, M. C., Knechtle, W. S., Balu, S., Sendak, M., & Theiling, J. (2021) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2021.02.029 · Cited by 49
Performance of Plasma Coproporphyrin I and III as OATP1B1 Biomarkers in Humans
Neuvonen, M., Tornio, A., Hirvensalo, P., Backman, J. T., & Niemi, M. (2021) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.2429 · Cited by 47
The MCC-F1 curve: a performance evaluation technique for binary classificationPreprint
Cao, C., Chicco, D., & Hoffman, M. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11278 · Cited by 91
Taneja, I., Damhorst, G. L., Lopez-Espina, C. G., Zhao, S. D., Zhu, R., Khan, S. A., White, K., Kumar, J. S., Vincent, A., Yeh, L., et al. (2021) · Clinical and Translational Science · doi:10.1111/cts.13030 · Cited by 51
Ebert, M. A., Gulliford, S., Acosta, O., de Crevoisier, R., McNutt, T., Heemsbergen, W. D., Witte, M. G., Palma, G., Rancati, T., & Fiorino, C. (2021) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ac0681 · Cited by 50
Li, W., & Guo, Q. (2021) · Ecology and Evolution · doi:10.1002/ece3.7826 · Cited by 49
Ogunleye, A. Z., Piyawajanusorn, C., Gonçalvès, A., Ghislat, G., & Ballester, P. J. (2022) · Advanced Science · doi:10.1002/advs.202201501 · Cited by 47
MacNell, N., Feinstein, L., Wilkerson, J., Salo, P. M., Molsberry, S. A., Fessler, M. B., Thorne, P. S., Motsinger‐Reif, A. A., & Zeldin, D. C. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280387 · Cited by 58
Cazares, T. A., Rizvi, F. W., Iyer, B., Chen, X., Kotliar, M., Bejjani, A. T., Wayman, J. A., Donmez, O. A., Wronowski, B., Parameswaran, S., et al. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010863 · Cited by 45
Vaid, A., Sawant, A., Suarez-Farinas, M., Lee, J., Kaul, S., Kovatch, P., Freeman, R., Jiang, J., Jayaraman, P., Fayad, Z., et al. (2023) · Annals of Internal Medicine · doi:10.7326/m23-0949 · Cited by 44
Machine Learning Methods for Credit Card Fraud Detection: A Survey
Dastidar, K. G., Caelen, O., & Granitzer, M. (2024) · IEEE Access · doi:10.1109/access.2024.3487298 · Cited by 78
AI for Data Quality Auditing: Detecting Mislabeled Work Zone Crashes Using Large Language Models
Jaradat, S., Acharya, N. K., Shivshankar, S., Alhadidi, T. I., & Elhenawy, M. M. Z. (2025) · Algorithms · doi:10.3390/a18060317 · Cited by 44
Predict Pairwise Trust Based on Machine Learning in Online Social Networks: A Survey
Liu, S., Zhang, L., & Yan, Z. (2018) · IEEE Access · doi:10.1109/access.2018.2869699 · Cited by 46
Biological relevance of computationally predicted pathogenicity of noncoding variants
Liu, L., Sanderford, M. D., Patel, R., Chandrashekar, P. B., Gibson, G. C., & Kumar, S. (2019) · Nature Communications · doi:10.1038/s41467-018-08270-y · Cited by 49
Scanning tunneling state recognition with multi-class neural network ensembles
Gordon, O., D’Hondt, P., Knijff, L., Freeney, S. E., Junqueira, F. L. Q., Moriarty, P., & Swart, I. (2019) · Review of Scientific Instruments · doi:10.1063/1.5099590 · Cited by 50
Qian, X., Zhang, Q., Liu, S., Wang, Y., Chen, X., Liu, J., Yang, Y., Chen, X., Wei, Y., Xiao, Q., et al. (2020) · European Radiology · doi:10.1007/s00330-019-06610-0 · Cited by 49
McMahon, D. E., Urza, A. K., Brown, J. L., Phelan, C., & Chambers, J. C. (2021) · Diversity and Distributions · doi:10.1111/ddi.13232 · Cited by 54
Khanal, J., Tayara, H., Zou, Q., & Chong, K. T. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.03.015 · Cited by 46
Evaluating the performance of drug-repurposing technologies
Schuler, J., Falls, Z., Mangione, W., Hudson, M. L., Bruggemann, L., & Samudrala, R. (2021) · Drug Discovery Today · doi:10.1016/j.drudis.2021.08.002 · Cited by 46
Automatic recognition of parasitic products in stool examination using object detection approach
Naing, K. M., Boonsang, S., Chuwongin, S., Kittichai, V., Tongloy, T., Prommongkol, S., Dekumyoy, P., & Watthanakulpanich, D. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.1065 · Cited by 46
Automating the assessment of biofouling in images using expert agreement as a gold standard
Mannix, E., Wei, S., Woodham, B. A., Wilkinson, P., & Robinson, A. P. (2021) · Scientific Reports · doi:10.1038/s41598-021-81011-2 · Cited by 49
Engeseth, G. M., He, R., Mirković, D., Yepes, P. P., Mohamed, A., Stieb, S., Fuller, C. D., Wu, R., Zhang, X., Hysing, L. B., et al. (2021) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2021.06.016 · Cited by 43
Usefulness of Random Forest Algorithm in Predicting Severe Acute Pancreatitis
Hong, W., Lu, Y., Zhou, X., Jin, S., Pan, J., Lin, Q., Yang, S., Basharat, Z., Zippi, M., & Goyal, H. (2022) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2022.893294 · Cited by 51
Graeßner, M., Jungwirth, B., Frank, E., Schaller, S. J., Kochs, E. F., Ulm, K., Blobner, M., Ulm, B., Podtschaske, A. H., & Kagerbauer, S. M. (2023) · Scientific Reports · doi:10.1038/s41598-023-33981-8 · Cited by 49
A phenotype centric benchmark of variant prioritisation tools
Anderson, D., & Lassmann, T. (2018) · npj Genomic Medicine · doi:10.1038/s41525-018-0044-9 · Cited by 43
Iorfino, F., Ho, N., Carpenter, J. S., Cross, S. P. M., Davenport, T. A., Hermens, D. F., Yee, H. F., Nichles, A., Zmicerevska, N., Guastella, A. J., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0243467 · Cited by 48
A Review of Tree-Based Approaches for Anomaly Detection
Barbariol, T., Chiara, F. D., Marcato, D., & Susto, G. A. (2021) · Springer series in reliability engineering · doi:10.1007/978-3-030-83819-5_7 · Cited by 56
Predicting Prolonged Length of ICU Stay through Machine Learning
Wu, J., Lin, Y., Li, P., Hu, Y., Zhang, L., & Kong, G. (2021) · Diagnostics · doi:10.3390/diagnostics11122242 · Cited by 62
Yang, J., Wang, L., Phadke, N. A., Wickner, P. G., Mancini, C. M., Blumenthal, K. G., & Zhou, L. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.22836 · Cited by 42
Wang, C.-Y., & Lin, J. J. H. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107721 · Cited by 77
A three-way decision ensemble method for imbalanced data oversampling
Yan, Y. T., Wu, Z. B., Du, X. Q., Chen, J., Zhao, S. G., & Zhang, Y. P. (2018) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2018.12.011 · Cited by 44
Dual binding mode of “bitter sugars” to their human bitter taste receptor target
Fierro, F., Giorgetti, A., Carloni, P., Meyerhof, W., & Alfonso‐Prieto, M. (2019) · Scientific Reports · doi:10.1038/s41598-019-44805-z · Cited by 49
Machine Learning based parameter tuning strategy for MMC based topology optimization
Jiang, X., Wang, H., Li, Y., & Mo, K. (2020) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2020.102841 · Cited by 44
Molecular machine learning with conformer ensembles
Axelrod, S., & Gómez‐Bombarelli, R. (2023) · Machine Learning Science and Technology · doi:10.1088/2632-2153/acefa7 · Cited by 76
Modeling multivariate clinical event time-series with recurrent temporal mechanisms
Lee, J. M., & Hauskrecht, M. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102021 · Cited by 54
The accuracy versus interpretability trade-off in fraud detection model
Nesvijevskaia, A., Ouillade, S., Guilmin, P., & Zucker, J. (2021) · Data & Policy · doi:10.1017/dap.2021.3 · Cited by 68
Puttanawarut, C., Sirirutbunkajorn, N., Tawong, N., Jiarpinitnun, C., Khachonkham, S., Pattaranutaporn, P., & Wongsawat, Y. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.768152 · Cited by 44
Automatic Annotation of Subsea Pipelines Using Deep Learning
Stamoulakatos, A., Cardona, J., McCaig, C., Murray, D., Filius, H., Atkinson, R. C., Bellekens, X., Michie, C., Andonović, I., Lazaridis, P. I., et al. (2020) · Sensors · doi:10.3390/s20030674 · Cited by 44
Wang, L., Tan, Y., Yang, X., Kuang, L., & Ping, P. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac080 · Cited by 54
Song, J., Lee, Y. C., & Lee, J. (2022) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-022-01981-6 · Cited by 47
Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future
Klie, J.-C., Webber, B., & Gurevych, I. (2023) · Computational Linguistics · doi:10.1162/coli_a_00464 · Cited by 104
Pan, Z., Zhang, R., Shen, S., Lin, Y., Zhang, L., Wang, X., Ye, Q., Wang, X., Chen, J., Zhao, Y., et al. (2023) · eBioMedicine · doi:10.1016/j.ebiom.2023.104443 · Cited by 43
Statsenko, Y. A., Babushkin, V., Talako, T., Kurbatova, T., Smetanina, D., Simiyu, G. L., Habuza, T., Ismail, F., Almansoori, T. M., Gorkom, K., et al. (2023) · Biomedicines · doi:10.3390/biomedicines11092370 · Cited by 49
Genome-scale annotation of protein binding sites via language model and geometric deep learning
Yuan, Q., Tian, C., & Yang, Y. (2024) · eLife · doi:10.7554/elife.93695.3 · Cited by 9
Bollack, A., Collij, L. E., García, D. V., Shekari, M., Altomare, D., Payoux, P., Dubois, B., Grau‐Rivera, O., Boada, M., Marquié, M., et al. (2024) · Alzheimer s & Dementia · doi:10.1002/alz.13761 · Cited by 50
Anomaly Detection in Road Networks Using Sliding-Window Tensor Factorization
Xu, M., Wu, J., Wang, H., & Cao, M. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2941649 · Cited by 41
Identifying risk of opioid use disorder for patients taking opioid medications with deep learning
Dong, X., Deng, J., Rashidian, S., Abell-Hart, K., Hou, W., Rosenthal, R. N., Saltz, M. M., Saltz, J. H., & Wang, F. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab043 · Cited by 47
Sung, S., Hung, L.-C., & Hu, Y. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104505 · Cited by 46
Defending Against Adversarial Attacks On Medical Imaging Ai System, Classification Or Detection?
Li, X., Pan, D., & Zhu, D. (2021) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi48211.2021.9433761 · Cited by 48
Leveraging Activity Recognition to Enable Protective Behavior Detection in Continuous Data
Wang, C., Gao, Y., Mathur, A., De C. Williams, A. C., Lane, N. D., & Bianchi-Berthouze, N. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463508 · Cited by 41
Nakamura, Y., Hanaoka, S., Nomura, Y., Nakao, T., Miki, S., Watadani, T., Yoshikawa, T., Hayashi, N., & Abe, O. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01623-6 · Cited by 41
Song, J., Hobensack, M., Bowles, K. H., McDonald, M. V., Cato, K., Rossetti, S. C., Chae, S., Kennedy, E. E., Barrón, Y., Sridharan, S., et al. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104039 · Cited by 46
Huang, J., Wu, X., Ling, S., Li, X., Wu, Y., Peng, L., & He, Z. (2022) · Environmental Science and Pollution Research · doi:10.1007/s11356-022-23732-z · Cited by 46
Credit card fraud detection using machine learning: A surveyPreprint
Lucas, Y., & Jurgovsky, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06479 · Cited by 150
Subseasonal Statistical Forecasts of Eastern U.S. Hot Temperature Events
Vijverberg, S., Schmeits, M., van der Wiel, K., & Coumou, D. (2020) · Monthly Weather Review · doi:10.1175/mwr-d-19-0409.1 · Cited by 40
Machine learning-based prediction of critical illness in children visiting the emergency department
Hwang, S., & Lee, B. (2022) · PLoS ONE · doi:10.1371/journal.pone.0264184 · Cited by 45
Metadata
- Authors
- Saito, T., & Rehmsmeier, M.
- Affiliations
- University of Bergen (NO)
- Corresponding author
- Saito, Takaya
- Rehmsmeier, Marc
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-03-04
- Volume
- 10
- Issue
- 3
- Pages
- e0118432
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4349800
- OpenAlex
- W1966716734
- Cited by
- 5,309
- References
- 56
- Citations by year
- 2026: 1482 · 2025: 762 · 2024: 528 · 2023: 505 · 2022: 551 · 2021: 563 · 2020: 402 · 2019: 283 · 2018: 133 · 2017: 64 · 2016: 27 · 2015: 3
- FWCI
- 74.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Universitetet i Bergen
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- class imbalance, binary classification, classifier evaluation, specificity, positive predictive value, sensitivity, bioinformatics classification, visual interpretability
- MeSH terms
- Classification, ROC Curve, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.