The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets
Open accessThe DOI matches its Crossref record; 55 of the 56 references it cites were found in a registry or an index.
Abstract
Binary classifiers are routinely evaluated with performance measures such as sensitivity and specificity, and performance is frequently illustrated with Receiver Operating Characteristics (ROC) plots. Alternative measures such as positive predictive value (PPV) and the associated Precision/Recall (PRC) plots are used less frequently. Many bioinformatics studies develop and evaluate classifiers that are to be applied to strongly imbalanced datasets in which the number of negatives outweighs the number of positives significantly. While ROC plots are visually appealing and provide an overview of a classifier's performance across a wide range of specificities, one can ask whether ROC plots could be misleading when applied in imbalanced classification scenarios. We show here that the visual interpretability of ROC plots in the context of imbalanced datasets can be deceptive with respect to conclusions about the reliability of classification performance, owing to an intuitive but wrong interpretation of specificity. PRC plots, on the other hand, can provide the viewer with an accurate prediction of future classification performance due to the fact that they evaluate the fraction of true positives among positive predictions. Our findings have potential implications for the interpretation of a large number of studies that use ROC plots on imbalanced datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“which is a more informative measure than ROC analysis in imbalanced scenarios59.”
From K-STAMM: a knowledge-enhanced spatial - temporal attention model with multimodal fusion for pneumonia prediction (2026) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“When dealing with highly imbalanced data as is the case with fraud detection, AUC (and ROC curves) may be too optimistic and the Area under the Precision-Recall Curve (AUPRC) gives a more informative picture of a classifier's performance (Davis and Goadrich, 2006; Saito and Rehmsmeier, 2015; Fern´ andez et al., 2018).”
From Data engineering for fraud detection (2021) · cites this paper as Saito and Rehmsmeier, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Because extreme prediction is by definition an imbalanced classification problem, we base our analysis on precision-recall plots which are well-suited for this case [42].”
From Learning skillful medium-range global weather forecasting (2023) · cites this paper as [42]
In the citing paper
“Furthermore, careful consideration should be given when selecting evaluation metrics, as the precision-recall curve is more informative for imbalanced datasets than the most commonly used receiver-operating characteristic curve (true positive rate vs. false positive rate) (Fig. 5C).115”
From Use of artificial intelligence for liver diseases: A survey from the EASL congress 2024 (2024) · cites this paper as [115]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Bartel (2004) · doi:10.1016/s0092-8674(04)00045-5 · Cited by 34,714
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 3×
“Here, we review the theoretical background of commonly used evaluation measures, specifically ROC [24, 25], PRC [26], concentrated ROC (CROC) [27], and Cost Curves (CC) [28].”
BackgroundMethodThe meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
Bioconductor: open software development for computational biology and bioinformatics
Gentleman, R. C., Carey, V. J., Bates, D. M., Bolstad, B., Dettling, M., Dudoit, S., Ellis, B., Gautier, L., Ge, Y., Gentry, J., et al. (2004) · Genome Biology · doi:10.1186/gb-2004-5-10-r80 · Cited by 12,172 · cited 1×
“Several integrated machine learning and bioinformatics platforms such as WEKA [34] and Bioconductor [42, 43] also have basic functions or libraries for making ROC and PRC plots.”
MethodA training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
R: A Language for Data Analysis and Graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,220
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Cited by
Items 701–800 of 1,023
Prediction of Childhood Obesity from Nationwide Health Records
Rossman, H., Shilo, S., Barbash‐Hazan, S., Artzi, N. S., Hadar, E., Balicer, R. D., Feldman, B., Wiznitzer, A., & Segal, E. (2021) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2021.02.010 · Cited by 40
Finding Quasars behind the Galactic Plane. I. Candidate Selections with Transfer Learning
Fu, Y., Wu, X.-B., Yang, Q., Brown, A. G. A., Feng, X., Ma, Q., & Li, S. (2021) · The Astrophysical Journal Supplement Series · doi:10.3847/1538-4365/abe85e · Cited by 43
Geana, A., Barch, D. M., Gold, J. M., Carter, C. S., MacDonald, A. W., Ragland, J. D., Silverstein, S. M., & Frank, M. J. (2021) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2021.03.017 · Cited by 48
Credit Decision Support Based on Real Set of Cash Loans Using Integrated Machine Learning Algorithms
Ziemba, P., Becker, J., Becker, A., Radomska-Zalas, A., Pawluk, M., & Wierzba, D. (2021) · Electronics · doi:10.3390/electronics10172099 · Cited by 41
Xuan, P., Fan, M., Cui, H., Zhang, T., & Nakaguchi, T. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab453 · Cited by 37
Lien, F., Lin, H.-S., Wu, Y.-T., & Chiueh, T. (2022) · BMC Infectious Diseases · doi:10.1186/s12879-022-07223-7 · Cited by 40
Perimeter Intrusion Detection by Video Surveillance: A Survey
Lohani, D., Crispim-Junior, C. F., Barthélemy, Q., Bertrand, S., Robinault, L., & Tougne, L. (2022) · Sensors · doi:10.3390/s22093601 · Cited by 47
Semisupervised Clustering Approach for Pipe Failure Prediction with Imbalanced Data Set
Zali, R. B., Latifi, M., Javadi, A. A., & Farmani, R. (2023) · Journal of Water Resources Planning and Management · doi:10.1061/jwrmd5.wreng-6263 · Cited by 35
North American Hardwoods Identification Using Machine-Learning
Lopes, D. J. V., Burgreen, G. W., & Entsminger, E. D. (2020) · Forests · doi:10.3390/f11030298 · Cited by 35
Li, X., & Willems, P. (2019) · Urban Water Journal · doi:10.1080/1573062x.2020.1726409 · Cited by 36
Wu, Y., Liu, Q., Qiu, Y., & Xie, L. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1010367 · Cited by 34
“The performances of the micro-ROCAUC have similar trends for both datasets.”
BackgroundInfluentialZhao, X., Wu, J., Zhao, X., & Yin, M. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac548 · Cited by 39
Phosformer: an explainable transformer model for protein kinase-specific phosphorylation predictions
Zhou, Z., Yeung, W., Gravel, N., Salcedo, M., Soleymani, S., Li, S., & Kannan, N. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad046 · Cited by 44
Karabacak, M., & Margetis, K. D. (2023) · Cancers · doi:10.3390/cancers15030812 · Cited by 44
Shi, M., Yang, A., Lau, E. S. H., Luk, A. O. Y., Ronald, C. W., Kong, A. P. S., Wong, R. S. M., Chan, J. C. M., Chan, J. C. N., & Chow, E. (2024) · PLoS Medicine · doi:10.1371/journal.pmed.1004369 · Cited by 41
Mattera, G., Vozza, M., Polden, J., Nele, L., & Pan, Z. S. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-024-02507-y · Cited by 41
Mylonas, A., Keall, P., Booth, J. T., Shieh, C., Eade, T., Poulsen, P. R., & Nguyen, D. T. (2019) · Medical Physics · doi:10.1002/mp.13519 · Cited by 36
Carletti, M., Masiero, C., Beghi, A., & Susto, G. A. (2019) · Procedia Manufacturing · doi:10.1016/j.promfg.2020.01.031 · Cited by 42
Machine Learning Applications in Endocrinology and Metabolism Research: An Overview
Hong, N., Park, H., & Rhee, Y. (2020) · Endocrinology and Metabolism · doi:10.3803/enm.2020.35.1.71 · Cited by 46
MacNeil, L., Missan, S., Luo, J., Trappenberg, T. P., & LaRoche, J. (2021) · BMC Ecology and Evolution · doi:10.1186/s12862-021-01839-0 · Cited by 46
Thoral, P. J., Fornasa, M., de Bruin, D. P., Tonutti, M., Hovenkamp, H., Driessen, R. H., Girbes, A. R. J., Hoogendoorn, M., & Elbers, P. W. G. (2021) · Critical Care Explorations · doi:10.1097/cce.0000000000000529 · Cited by 34
Boosted decision trees in the era of new physics: a smuon analysis case study
Cornell, A. S., Doorsamy, W., Fuks, B., Harmsen, G. E., & Mason, L. H. (2022) · Journal of High Energy Physics · doi:10.1007/jhep04(2022)015 · Cited by 50
Laios, A., Katsenou, A., Tan, Y. S., Johnson, R. L., Otify, M., Kaufmann, A., Munot, S., Thangavelu, A., Hutson, R., Broadhead, T., et al. (2021) · Cancer Control · doi:10.1177/10732748211044678 · Cited by 36
Convolutional Neural Networks for Audio-Based Continuous Infant Cry Monitoring at Home
Xie, J., Long, X., Otte, R. A., & Shan, C. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3123906 · Cited by 37
Luetkens, J. A., Nowak, S., Mesropyan, N., Block, W., Praktiknjo, M., Chang, J., Bauckhage, C., Sifa, R., Sprinkart, A. M., Faron, A., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-12410-2 · Cited by 38
Robust Out-of-distribution Detection for Neural NetworksPreprint
Chen, J., Li, Y., Wu, X., Liang, Y., & Jha, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.09711 · Cited by 89
Use of artificial intelligence for liver diseases: A survey from the EASL congress 2024
Žigutytė, L., Sorz-Nechay, T., Clusmann, J., & Kather, J. N. (2024) · JHEP Reports · doi:10.1016/j.jhepr.2024.101209 · Cited by 33
Anc2vec: embedding gene ontology terms by preserving ancestors relationships
Edera, A., Milone, D. H., & Stegmayer, G. S. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac003 · Cited by 37
Reclaiming the Past to Transcend the Present: Nostalgic Appeals in U.S. Presidential Elections
Bonikowski, B., & Stuhler, O. (2022) · Sociological Forum · doi:10.1111/socf.12838 · Cited by 39
Passenger and heavy vehicle collisions with pedestrians: Assessment of injury mechanisms and risk
Schubert, A., Babisch, S., Scanlon, J. M., Campolettano, E. T., Rößler, R., Unger, T., & McMurry, T. L. (2023) · Accident Analysis & Prevention · doi:10.1016/j.aap.2023.107139 · Cited by 33
Empirical analysis of performance assessment for imbalanced classification
Gaudreault, J.-G., & Branco, P. (2024) · Machine Learning · doi:10.1007/s10994-023-06497-5 · Cited by 52
Carlo, A. D., Ronchi, D., Piastra, M., Tosca, E. M., & Magni, P. (2024) · Pharmaceutics · doi:10.3390/pharmaceutics16060776 · Cited by 40
Agonist Binding to Chemosensory Receptors: A Systematic Bioinformatics Analysis
Fierro, F., Suku, E., Alfonso‐Prieto, M., Giorgetti, A., Cichon, S., & Carloni, P. (2017) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2017.00063 · Cited by 36
Computational Models of Miscommunication Phenomena
Purver, M., Hough, J., & Howes, C. (2018) · Topics in Cognitive Science · doi:10.1111/tops.12324 · Cited by 42
Evaluating the Boosting Approach to Machine Learning for Formation Lithology Classification
Dev, V. A., & Eden, M. R. (2018) · Computer-aided chemical engineering/Computer aided chemical engineering · doi:10.1016/b978-0-444-64241-7.50239-1 · Cited by 33
DeepIsoFun: a deep domain adaptation approach to predict isoform functions
Shaw, D., Chen, H., & Jiang, T. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty1017 · Cited by 37
An efficient character recognition method using enhanced HOG for spam image detection
Naiemi, F., Ghods, V., & Khalesi, H. (2019) · Soft Computing · doi:10.1007/s00500-018-03728-z · Cited by 33
Automatic classification of complaint letters according to service provider categories
HaCohen‐Kerner, Y., Dilmon, R., Hone, M., & Ben-Basan, M. A. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2019.102102 · Cited by 35
Generative Oversampling with a Contrastive Variational Autoencoder
Dai, W., Ng, K., Severson, K., Huang, W., Anderson, F. A., & Stultz, C. M. (2019) · Proceedings · doi:10.1109/icdm.2019.00020 · Cited by 33
Self-Diagnosis of Multiphase Flow Meters through Machine Learning-Based Anomaly Detection
Barbariol, T., Feltresi, E., & Susto, G. A. (2020) · Energies · doi:10.3390/en13123136 · Cited by 33
Exploiting the full potential of Bayesian networks in predictive ecology
Ramazi, P., Kunegel‐Lion, M., Greiner, R., & Lewis, M. A. (2020) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13509 · Cited by 40
Contrastive learning improves critical event prediction in COVID-19 patients
Wanyan, T., Honarvar, H., Jaladanki, S. K., Zang, C., Naik, N., Somani, S. S., Freitas, J. K. D., Paranjpe, I., Vaid, A., Zhang, J., et al. (2021) · Patterns · doi:10.1016/j.patter.2021.100389 · Cited by 38
Hogue, S.-C., Chen, F., Brassard, G., Lebel, D., Bussières, J., Durand, A., & Thibault, M. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab071 · Cited by 37
Rummens, A., Snaphaan, T., de Weghe, N. V., den Poel, D. V., Pauwels, L. J. R., & Hardyns, W. (2021) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi10060369 · Cited by 62
Boni, Y. T., Cammarota, R. C., Liao, K., Sigman, M. S., & Davies, H. M. L. (2022) · Journal of the American Chemical Society · doi:10.1021/jacs.2c04383 · Cited by 34
Labriffe, M., Woillard, J., Gwinner, W., Braesen, J.-H., Anglicheau, D., Rabant, M., Koshy, P. J., Naesens, M., & Marquet, P. (2022) · American Journal of Transplantation · doi:10.1111/ajt.17192 · Cited by 36
Gattermann-Itschert, T., & Thonemann, U. W. (2022) · Industrial Marketing Management · doi:10.1016/j.indmarman.2022.09.023 · Cited by 63
Stojanov, D., Lazarova, E., Veljkova, E., Rubartelli, P., & Giacomini, M. L. (2023) · Journal of King Saud University - Science · doi:10.1016/j.jksus.2023.102573 · Cited by 33
Stable variable selection of class-imbalanced data with precision-recall criterion
Fu, G., Xu, F., Zhang, B.-Y., & Yi, L. (2017) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2017.10.015 · Cited by 34
Lo-Ciganic, W.-H., Donohue, J. M., Hulsey, E. G., Barnes, S., Li, Y., Kuza, C. C., Yang, Q., Buchanich, J., Huang, J. L., Mair, C., et al. (2021) · PLOS ONE · doi:10.1371/journal.pone.0248360 · Cited by 32
“8: very good) and precision-recall curves between two GBM models [50].”
BackgroundSVDNVLDA: predicting lncRNA-disease associations by Singular Value Decomposition and node2vec
Li, J., Li, J., Kong, M., Wang, D., Fu, K., & Shi, J. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04457-1 · Cited by 35
Machine learning prediction of academic collaboration networks
Resce, G., Zinilli, A., & Cerulli, G. (2022) · Scientific Reports · doi:10.1038/s41598-022-26531-1 · Cited by 35
Schultz, K., Bej, S., Hahn, W., Wolfien, M., Srivastava, P. K., & Wolkenhauer, O. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.110138 · Cited by 45
Kim, K., Kim, M., Adamopoulos, I. E., & Tagkopoulos, I. (2019) · Clinical Immunology · doi:10.1016/j.clim.2019.03.002 · Cited by 32
Baker, M. R., Palsson, W. A., Zimmermann, M., & Rooper, C. N. (2019) · Fisheries Oceanography · doi:10.1111/fog.12442 · Cited by 33
Tavakoli, H., Chen, W., Sin, D. D., FitzGerald, J. M., & Sadatsafavi, M. (2020) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202001-070oc · Cited by 32
Funke, T., & Becker, T. (2020) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2020.06.015 · Cited by 40
Chicco, D., & Oneto, L. (2021) · Health Informatics Journal · doi:10.1177/1460458220984205 · Cited by 33
Seki, T., Kawazoe, Y., & Ohe, K. (2021) · PLoS ONE · doi:10.1371/journal.pone.0246640 · Cited by 38
Credit Risk Model Based on Central Bank Credit Registry Data
Doko, F., Kalajdziski, S., & Mishkovski, I. (2021) · Journal of risk and financial management · doi:10.3390/jrfm14030138 · Cited by 41
External validation of an opioid misuse machine learning classifier in hospitalized adult patients
Afshar, M., Sharma, B., Bhalla, S., Thompson, H. M., Dligach, D., Boley, R. A., Kishen, E., Simmons, A., Perticone, K., & Karnik, N. S. (2021) · Addiction Science & Clinical Practice · doi:10.1186/s13722-021-00229-7 · Cited by 35
Smyth, B., Lawlor, A., Berndsen, J., & Feely, C. (2021) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-021-09299-3 · Cited by 43
A machine learning model for lapse prediction in life insurance contracts
Azzone, M., Barucci, E., Moncayo, G. G., & Marazzina, D. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116261 · Cited by 50
The pneumonia severity index: Assessment and comparison to popular machine learning classifiers
Wang, D., Willis, D. R., & Yih, Y. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104778 · Cited by 36
Virtual ChIP-seq: predicting transcription factor binding by learning from the transcriptome
Karimzadeh, M., & Hoffman, M. M. (2022) · Genome Biology · doi:10.1186/s13059-022-02690-2 · Cited by 32
Bracher‐Smith, M., Rees, E., Menzies, G. E., Walters, J., O’Donovan, M. C., Owen, M. J., Kirov, G., & Escott‐Price, V. (2022) · Schizophrenia Research · doi:10.1016/j.schres.2022.06.006 · Cited by 47
Yu, H., & Mooney, M. A. (2022) · Computers and Geotechnics · doi:10.1016/j.compgeo.2022.105159 · Cited by 33
Sctensor detects many-to-many cell–cell interactions from single cell RNA-sequencing data
Tsuyuzaki, K., Ishii, M., & Nikaido, I. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05490-y · Cited by 80
Hauben, M., Aronson, J. K., & Ferner, R. E. (2016) · Drug Safety · doi:10.1007/s40264-016-0392-2 · Cited by 32
Fake News Classification Based on Subjective Language
Jerônimo, C. L. M., Marinho, L. B., Campelo, C. E. C., Veloso, A., & Melo, A. S. D. C. (2019) · International Conference on Information Integration and Web-based Applications & Services (iiWAS) · doi:10.1145/3366030.3366039 · Cited by 43
Tiulpin, A., Klein, S., Bierma-Zeinstra, S. M. A., Thevenot, J., Rahtu, E., Meurs, J. V., Oei, E. H. G., & Saarakkala, S. (2019) · Scientific Reports · doi:10.1038/s41598-019-56527-3 · Cited by 31
Ahamed, M. F., Syfullah, M. K., Sarkar, O., Islam, M. T., Nahiduzzaman, M., Islam, M. R., Khandakar, A. A., Ayari, M. A., & Chowdhury, M. E. H. (2023) · Sensors · doi:10.3390/s23187724 · Cited by 34
Wang, W., Sheng, R., Liao, S., Wu, Z., Wang, L., Liu, C., Yang, C., & Jiang, R. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01172-0 · Cited by 35
Lin, T., Lu, C., Karmakar, R., Nampalley, K., Mukundan, A., Hsiao, Y., Hsieh, S.-C., & Wang, H. (2024) · Diagnostics · doi:10.3390/diagnostics14151672 · Cited by 31
Large-scale Dataset Pruning with Dynamic Uncertainty
He, M., Yang, S., Huang, T., & Zhao, B. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00767 · Cited by 99
Reciprocal Perspective for Improved Protein-Protein Interaction Prediction
Dick, K., & Green, J. R. (2018) · Scientific Reports · doi:10.1038/s41598-018-30044-1 · Cited by 32
Sleep monitoring with the Apple Watch: comparison to a clinically validated actigraph
Roomkham, S., Hittle, M., Cheung, J., Lovell, D. R., Mignot, E. J.-M., & Perrin, D. (2019) · F1000Research · doi:10.12688/f1000research.19020.1 · Cited by 30
Dijkstra, L., Garling, M., Foraita, R., & Pigeot, I. (2020) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4970 · Cited by 35
Improving Money Laundering Detection Using Optimized Support Vector Machine
Pambudi, B. N., Hidayah, I., & Fauziati, S. (2019) · International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) · doi:10.1109/isriti48646.2019.9034655 · Cited by 31
Guidelines for effective evaluation and comparison of wildland fire occurrence prediction models
Phelps, N., & Woolford, D. G. (2021) · International Journal of Wildland Fire · doi:10.1071/wf20134 · Cited by 31
Brent, D. A., Grupp‐Phelan, J., O'Shea, B. A., Patel, S. J., Mahabee‐Gittens, E. M., Rogers, A. J., Duffy, S. J., Shenoi, R., Chernick, L. S., Casper, T. C., et al. (2021) · Psychological Medicine · doi:10.1017/s0033291721001215 · Cited by 35
Revisiting performance metrics for prediction with rare outcomes
Adhikari, S., Normand, S. T., Bloom, J. P., Shahian, D. M., & Rose, S. (2021) · Statistical Methods in Medical Research · doi:10.1177/09622802211038754 · Cited by 35
Stefan, C. P., Hall, A. T., Graham, A. S., & Minogue, T. D. (2022) · Journal of Molecular Diagnostics · doi:10.1016/j.jmoldx.2021.12.005 · Cited by 39
Călburean, P., Grebenișan, P., Nistor, I.-A., Pál, K., Vacariu, V., Drincal, R.-K., Țepeș, O., Bârlea, I., Şuş, I., Somkereki, C., et al. (2022) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2022.03.028 · Cited by 36
Etu, E.-E., Monplaisir, L. F., Arslanturk, S., Masoud, S., Aguwa, C. C., Markevych, I., & Miller, J. B. (2022) · IEEE Access · doi:10.1109/access.2022.3168045 · Cited by 31
Islam, T., Meade, N., Carson, R. T., Louviere, J. J., & Wang, J. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.07.004 · Cited by 43
Barukab, O., Ahmad, A., Khan, T., & Kunhumuhammed, M. R. T. (2022) · Diagnostics · doi:10.3390/diagnostics12123000 · Cited by 39
Schmidt, J., & Moss, R. E. S. (2021) · Computers and Geotechnics · doi:10.1016/j.compgeo.2020.103963 · Cited by 30
Ru, B., Tan, X., Liu, Y., Kannapur, K., Ramanan, D., Kessler, G. N., Lautsch, D., & Fonarow, G. C. (2023) · JMIR Formative Research · doi:10.2196/41775 · Cited by 33
4DST-BTMD: An Infrared Small Target Detection Method Based on 4-D Data-Sphered Space
Luo, Y., Li, X., Chen, S., & Xia, C. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3347420 · Cited by 30
Xu, D., & Xu, Z. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102950 · Cited by 35
Automated Deep Learning Phenotyping of Tricuspid Regurgitation in Echocardiography
Vrudhula, A., Vukadinovic, M., Haeffele, C. L., Kwan, A. C., Berman, D. S., Liang, D. H., Siegel, R., Cheng, S. C., & Ouyang, D. (2025) · JAMA Cardiology · doi:10.1001/jamacardio.2025.0498 · Cited by 41
Nagashima, T., Shimoda, Y., TANABE, T., Naruoka, A., Saito, J., Serizawa, M., Ohshima, K., Urakami, K., Ohnami, S., Ohnami, S., et al. (2016) · Biomedical Research · doi:10.2220/biomedres.37.359 · Cited by 33
Rehimi, R., Nikolić, M., Cruz-Molina, S., Tebartz, C., Frommolt, P., Mahabir, E., Clément‐Ziza, M., & Rada-Iglesias, Á. (2016) · Cell Reports · doi:10.1016/j.celrep.2016.11.046 · Cited by 28
On Obstructing Obscenity Obfuscation
Rojas–Galeano, S. (2017) · ACM Transactions on the Web · doi:10.1145/3032963 · Cited by 30
Rao, R. R., & Makkithaya, K. (2017) · International Journal of Electrical and Computer Engineering (IJECE) · doi:10.11591/ijece.v7i4.pp2215-2222 · Cited by 30
Multi-channel PINN: investigating scalable and transferable neural networks for drug discovery
Lee, M., Kim, H., Joe, H., & Kim, H. (2019) · Journal of Cheminformatics · doi:10.1186/s13321-019-0368-1 · Cited by 35
Unsupervised Boosting-Based Autoencoder Ensembles for Outlier Detection
Sarvari, H., Domeniconi, C., Prenkaj, B., & Stilo, G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-75762-5_8 · Cited by 34
Miao, F., Carpenter, N. S., Wang, Z., Holcomb, A. S., & Woolery, E. W. (2020) · Seismological Research Letters · doi:10.1785/0220190084 · Cited by 30
Synthetic Oversampling of Multi-label Data Based on Local Label Distribution
Liu, B., & Tsoumakas, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46147-8_11 · Cited by 30
Metadata
- Authors
- Saito, T., & Rehmsmeier, M.
- Affiliations
- University of Bergen (NO)
- Corresponding author
- Saito, Takaya
- Rehmsmeier, Marc
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-03-04
- Volume
- 10
- Issue
- 3
- Pages
- e0118432
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4349800
- OpenAlex
- W1966716734
- Cited by
- 5,309
- References
- 56
- Citations by year
- 2026: 1482 · 2025: 762 · 2024: 528 · 2023: 505 · 2022: 551 · 2021: 563 · 2020: 402 · 2019: 283 · 2018: 133 · 2017: 64 · 2016: 27 · 2015: 3
- FWCI
- 74.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Universitetet i Bergen
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- class imbalance, binary classification, classifier evaluation, specificity, positive predictive value, sensitivity, bioinformatics classification, visual interpretability
- MeSH terms
- Classification, ROC Curve, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.