PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration
Open accessThe DOI matches its Crossref record; 194 of the 201 references it cites were found in a registry or an index.
Abstract
Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMeasuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Research works printed but not linked: 2
- 10.1177/0962280218785504.30032705 · doi:10.1177/0962280218785504.30032705
- Wong SS, 2003, AMIA Annu Symp Proc, 728
Cited by
Items 301–400 of 1,003
Chandna, A., Tan, R., Carter, M. J., Bruel, A. V. D., Verbakel, J. Y. J., Koshiaris, C., Salim, N., Lubell, Y., Turner, P., & Keitel, K. (2021) · BMJ Global Health · doi:10.1136/bmjgh-2020-003451 · Cited by 25
Systematic Review of Prognosis Models in Predicting Tooth Loss in Periodontitis
Chow, D. Y., Tay, J. R. H., & Nascimento, G. G. (2024) · Journal of Dental Research · doi:10.1177/00220345241237448 · Cited by 26
“Studies that validated existing models by reestimating regression weights were assessed as model development studies (Moons et al. 2019).”
BackgroundMethodResultInfluentialPinto, C. M., Sánchez-Tecuatl, M., Berra‐Romani, R., Castro, I. D. M., Etchegaray‐Morales, I., Munguía‐Realpozo, P., Cárdenas-García, M., Arellano-Avendaño, F. J., & García‐Carrasco, M. (2024) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2024.152501 · Cited by 26
Risk prediction models for adolescent suicide: A systematic review and meta-analysis
Li, R., Yue, Y., Gu, X., Xiong, L., Luo, M., & Li, L. (2025) · Psychiatry Research · doi:10.1016/j.psychres.2025.116405 · Cited by 30
Cama-Olivares, A., Braun, C. G., Takeuchi, T., O’Hagan, E. C., Kaiser, K. A., Ghazi, L., Chen, J., Forni, L. G., Kane‐Gill, S. L., Ostermann, M., et al. (2025) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000702 · Cited by 31
Vermeulen, R. J., Andersson, V., Banken, J., Hannink, G., Govers, T. M., Rovers, M. M., & Rikkert, M. O. (2025) · Alzheimer s & Dementia · doi:10.1002/alz.70069 · Cited by 29
Parkin, C. J., Moritz, P., Kirkland, O., & Glover, A. R. (2021) · Journal of Surgical Research · doi:10.1016/j.jss.2021.07.009 · Cited by 27
Wang, H., Wu, W. K., Han, C., Zheng, J., Cai, X., Chang, S., Shi, J., Xu, N., & Ai, Z. (2021) · JMIR Medical Informatics · doi:10.2196/30079 · Cited by 27
Prediction Models for Gastric Cancer Risk in the General Population: A Systematic Review
Gu, J., Chen, R., Wang, S., Li, M., Fan, Z., Li, X., Zhou, J., Sun, K., & Wei, W. (2022) · Cancer Prevention Research · doi:10.1158/1940-6207.capr-21-0426 · Cited by 28
Petersen, G. C., Shatalov, J., Verma, T., Brim, W. R., Subramanian, H., Brackett, A., Bahar, R. C., Merkaj, S., Zeevi, T., Staib, L. H., et al. (2022) · American Journal of Neuroradiology · doi:10.3174/ajnr.a7473 · Cited by 26
Atkins, D., Makridis, C. A., Alterovitz, G., Ramoni, R. B., & Clancy, C. M. (2022) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-122220-110053 · Cited by 25
Koch, A. H., Jeelof, L. S., Muntinga, C. L. P., Gootzen, T. A., van de Kruis, N. M. A., Nederend, J., Boers, T. G. W., van der Sommen, F., & Piek, J. M. J. (2023) · Insights into Imaging · doi:10.1186/s13244-022-01345-x · Cited by 26
Gou, H., Song, H., Tian, Z., & Liu, Y. (2023) · Preventive Medicine · doi:10.1016/j.ypmed.2023.107823 · Cited by 33
Comparison of in-hospital mortality risk prediction models from COVID-19
El-Solh, A., Lawson, Y., Carter, M., El-Solh, D. A., & Mergenhagen, K. A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0244629 · Cited by 29
Li, Y., Jin, N., Zhan, Q., Huang, Y., Sun, A., Yin, F., Li, Z., Hu, J., & Liu, Z. (2025) · Frontiers in Endocrinology · doi:10.3389/fendo.2025.1495306 · Cited by 35
Predictive models in extracorporeal membrane oxygenation (ECMO): a systematic review
Giordano, L., Francavilla, A., Bottio, T., Dell’Amore, A., Gregori, D., Navalesi, P., Lorenzoni, G., & Baldi, I. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02211-7 · Cited by 25
Yoong, S. Q., Porock, D. C., Whitty, D., Tam, W., & Zhang, H. (2023) · Palliative Medicine · doi:10.1177/02692163231180657 · Cited by 24
Appel, K. S., Geisler, R., Maier, D., Miljukov, O., Pütz, S. M., & Vehreschild, J. J. (2023) · Clinical Infectious Diseases · doi:10.1093/cid/ciad618 · Cited by 21
“A “high” rating within at least 1 question or subdomain leads to an overall ROB of “high” [15].”
BackgroundMethodInfluentialPrediction of Morbidity and Mortality After Esophagectomy: A Systematic Review
Amerongen, M. P. V. N., de Grooth, H., Veerman, G., Ziesemer, K. A., Henegouwen, M. I. V. B., & Tuinman, P. R. (2024) · Annals of Surgical Oncology · doi:10.1245/s10434-024-14997-4 · Cited by 32
“19 This tool, especially designed for systematic reviews of prediction models, assesses the risk of bias in four domains (participants, predictors, outcome, and statistical analysis) and addresses the concerns of applicability in three domains (participants, predictors, and outcome).”
BackgroundMethodAmanollahi, M., Jameie, M., Looha, M. A., Basti, F. A., Cattarinussi, G., Moghaddam, H. S., Camillo, F. D., Akhondzadeh, S., Pigoni, A., Sambataro, F., et al. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.06.061 · Cited by 27
Risk prediction models for frailty in older adults: A systematic review and critical appraisal
Kong, L., Li, Y., Lyu, Q., Liu, D., & Yang, J. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105068 · Cited by 22
Nayebirad, S., Hassanzadeh, A., Vahdani, A. M., Mohamadi, A., Forghani, S., Shafiee, A., & Masoudkabir, F. (2025) · BMC Cardiovascular Disorders · doi:10.1186/s12872-025-04746-0 · Cited by 26
“Therefore, PROBAST recommends against the use of simple data splitting into train and validation sets, while encouraging using cross-validation [77–79].”
FINDRISC in Latin America: a systematic review of diagnosis and prognosis models
Carrillo-Larco, R. M., Aparcana-Granda, D. J., Mejia, J. R., & Bernabé‐Ortiz, A. (2020) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2019-001169 · Cited by 27
Ashford, M. T., Veitch, D. P., Neuhaus, J. M., Nosheny, R. L., Tosun, D., & Weiner, M. W. (2021) · Alzheimer s & Dementia · doi:10.1002/alz.12253 · Cited by 23
Prediction Models for Prognosis of Cervical Cancer: Systematic Review and Critical Appraisal
He, B., Chen, W., Liu, L., Hou, Z., Zhu, H., Cheng, H., Zhang, Y., Zhan, S., & Wang, S. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.654454 · Cited by 28
Deawjaroen, K., Sillabutra, J., Poolsup, N., Stewart, D. C., & Suksomboon, N. (2021) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15104 · Cited by 26
Pivodic, A., Smith, L. E. H., Hård, A., Löfqvist, C., Almeida, A. C., Al‐Hawasi, A., Larsson, E. K. B., Lundgren, P., Sunnqvist, B., Törnqvist, K., et al. (2022) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2021-320738 · Cited by 23
Filipow, N., Main, E., Sebire, N. J., Booth, J., Taylor, A. M., Davies, G., & Stanojevic, S. (2022) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2021-001165 · Cited by 29
Raveendran, D., Penny‐Dimri, J. C., Segal, R., Smith, J. A., Plummer, M. P., Liu, Z., & Perry, L. A. (2022) · Journal of Cardiothoracic Surgery · doi:10.1186/s13019-022-01870-2 · Cited by 26
Reproducibility of prediction models in health services research
Belbasis, L., & Panagiotou, O. A. (2022) · BMC Research Notes · doi:10.1186/s13104-022-06082-4 · Cited by 28
Li, L., Lu, Y., Yang, L. L., Xu, W., & Yu, J. (2022) · Journal of Clinical Nursing · doi:10.1111/jocn.16503 · Cited by 25
Guerrero‐Ramos, F., Subiela, J. D., Faba, O. R., Aumatell, J., Manfredi, C., Bozzini, G., Otero, J., & Couñago, F. (2022) · Bladder Cancer · doi:10.3233/blc-220055 · Cited by 23
“The quality of the included studies was analyzed independently by three reviewers (FGR, JDS, and ORF) using the prediction model risk of bias assessment tool (PROBAST) [10].”
BackgroundMethodInfluentialElwenspoek, M. M. C., Thom, H. H. Z., Sheppard, A. L., Keeney, E., O’Donnell, R., Jackson, J., Roadevin, C., Dawson, S., Lane, D., Stubbs, J., et al. (2022) · Health Technology Assessment · doi:10.3310/zuce8371 · Cited by 30
Self-citationMachine learning approach to predict postpartum haemorrhage: a systematic review protocol
Boujarzadeh, B., Ranjbar, A., Banihashemi, F., Mehrnoush, V., Darsareh, F., & Saffari, M. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-067661 · Cited by 23
Assessing Disease Activity in Pediatric Crohn’s Disease Using Ultrasound
van Wassenaer, E. A., van Rijn, R. R., de Voogd, F. A. E., van Schuppen, J., Kindermann, A., de Meij, T. G. J., Limbergen, J. V., Gecse, K. B., D’Haens, G., Benninga, M. A., et al. (2023) · Journal of Pediatric Gastroenterology and Nutrition · doi:10.1097/mpg.0000000000003727 · Cited by 23
“In addition, according to the Prediction model study Risk Of Bias Assessment Tool (30) predictors of a model should ideally not be based on univariate analyses, but rather on existing knowledge.”
MethodInfective endocarditis: Do we have an effective risk score model? A systematic review
Rizzo, V., Salmasi, M. Y., Sabetai, M. M., Primus, C., Sandoe, J. A. T., Lewis, M. E., Woldman, S., & Athanasiou, T. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2023.1093363 · Cited by 30
Shi, L., Sheng, M., Wei, Z., Liu, L., & Zhao, J. (2023) · Academic Radiology · doi:10.1016/j.acra.2023.05.026 · Cited by 22
Greig, L., Aspe, R., Hall, A., Comfort, P., Cooper, K., & Swinton, P. A. (2023) · Sports Medicine · doi:10.1007/s40279-023-01854-9 · Cited by 32
“Risk-of-bias assessment for this review was completed using a modified version of the Prediction model Risk Of Bias ASsessment Tool (PROBAST), which is a validated risk-of-bias tool used when studies are designed to assess the predictive validity of statistical models [33].”
MethodTesting Clinical Prediction Models
Wang, J., & Li, Y. (2020) · JAMA · doi:10.1001/jama.2020.19389 · Cited by 22
Shen, L., An, J., Wang, N., Wu, J., Yao, J., & Gao, Y. (2024) · World Journal of Urology · doi:10.1007/s00345-024-05145-4 · Cited by 23
“We further performed quality and risk of bias assessment using Prediction Model Risk of Bias Assessment Tool [11] (PROBAST).”
Martin, G. P., Riley, R. D., Ensor, J., & Grant, S. W. (2025) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezaf142 · Cited by 35
“Additionally, there is guidance to aid the critical appraisal of CPM studies [47, 48], and guidance on reporting CPM studies through the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) checklist, and associated explanation and elaboration…”
Self-citationMachine learning algorithms for heart disease diagnosis: A systematic review
Mao, Y., Jimma, B. L., & Belsty, T. (2025) · Current Problems in Cardiology · doi:10.1016/j.cpcardiol.2025.103082 · Cited by 34
Large Language Models in Critical Care Medicine: Scoping Review
Shi, T., Ma, J., Yu, Z., Xu, H., Yang, R., Xiong, M., Xiao, M., Li, Y., Zhao, H., & Kong, G. (2025) · JMIR Medical Informatics · doi:10.2196/76326 · Cited by 21
“We critically appraised all included studies using the PROBAST-AI (Prediction Model Risk of Bias Assessment Tool-Artificial Intelligence) [67], rating Risk of Bias (RoB) across 4 domains (participants, predictors, outcome, and analysis) and applicability across 3 domains (participants, predictors,…”
Muttalib, F., Clavel, V., Yaeger, L., Shah, V. S., & Adhikari, N. K. J. (2020) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2020.05.016 · Cited by 29
Risk Prediction Models for Inadvertent Intraoperative Hypothermia: A Systematic Review
Yan, L., Yao, L., Zhao, Q., Xiao, M., Li, Y., & Min, S. (2021) · Journal of PeriAnesthesia Nursing · doi:10.1016/j.jopan.2021.02.011 · Cited by 23
Vannucci, M., Laracca, G. G., Mercantini, P., Perretta, S., Padoy, N., Dallemagne, B., & Mascagni, P. (2021) · Surgery · doi:10.1016/j.surg.2021.10.001 · Cited by 31
Langenberger, B., Thoma, A., & Vogt, V. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01751-7 · Cited by 22
Poénou, G., Toledano, E., Helfer, H., Plaisance, L., Happe, F., Versini, E., Diab, N., Djennaoui, S., & Mahé, I. (2022) · Cancers · doi:10.3390/cancers14081937 · Cited by 22
Validity of three risk prediction models for dementia or cognitive impairment in Australia
Geethadevi, G. M., Peel, R., Bell, J. S., Cross, A. J., Hancock, S., Ilomäki, J., Tang, T., Attia, J. R., & George, J. (2022) · Age and Ageing · doi:10.1093/ageing/afac307 · Cited by 22
“We did not categorise the continuous variables to reduce their loss of predictive ability [35].”
Mertens, E., Barrenechea-Pulache, A., Sagastume, D., Vasquez, M. S., Vandevijvere, S., & Peñalvo, J. L. (2023) · BMC Cancer · doi:10.1186/s12885-023-11174-w · Cited by 29
“Bias assessment was performed in parallel to data extraction, also in duplicate, and for both model development and validation, following the framework of PROBAST (Prediction model Risk Of Bias ASsessment Tool) [24], that allows the classification of each study as having a high, unclear or low risk of bias according t…”
MethodPeng, Z.-H., Tian, J., Chen, B., Zhou, H., Bi, H., He, M., Li, M., Zheng, X., Wang, Y., Chong, T., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-45804-x · Cited by 23
“…of the relationships between candidate predictors and the result is a preferable strategy for selecting candidate predictors in multivariable modeling, according to the current bias assessment criteria (PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies) 30 .”
MethodWang, Q., Chang, Z., Liu, X., Wang, Y. X., Feng, C., Ping, Y., & Feng, X. (2024) · Journal of Medical Internet Research · doi:10.2196/48527 · Cited by 32
“The Prediction Model Risk of Bias Assessment (PROBAST), a technique for predicting the model risk of bias, was used to assess the risk of bias in the predictive models reported in eligible studies [32,33].”
Liang, Q., Ding, S., Chen, J., Chen, X., Xu, Y., Xu, Z., & Huang, M. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02504-4 · Cited by 22
“We evaluate the bias of the prediction model through PROBAST framework and write the prediction model through the TRIPOD scheme to ensure the structural integrity of the research [22, 23].”
MethodWang, X., Zhou, S., Ye, N., Li, Y., Zhou, P., Gao, C., & Hu, H. Z. (2024) · BMC Geriatrics · doi:10.1186/s12877-024-05044-8 · Cited by 26
“…from the conventional Meta-analysis, due to the significant heterogeneity of the predictive models, we directly used the random effects model [13, 22], the AUC of the models and the calculated (approximate) 95% prediction intervals were combined and estimated, heterogeneity was quantified…”
MethodCastagno, S., Gompels, B., Strangmark, E., Robertson-Waters, E. E., Birch, M., van der Schaar, M., & McCaskie, A. W. (2024) · The Bone & Joint Journal · doi:10.1302/0301-620x.106b11.bjj-2024-0453.r1 · Cited by 23
Veirman, E., Ryckeghem, D. M. L. V., Paepe, A. L. D., Kirtley, O. J., & Crombez, G. (2019) · PAIN Reports · doi:10.1097/pr9.0000000000000775 · Cited by 22
Cooray, S. D., Wijeyaratne, L. A., Soldatos, G., Allotey, J., Boyle, J. A., & Teede, H. J. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17093048 · Cited by 27
Li, L., Fang, X., Cheng, L., Wang, P., Li, S., Yu, H., Zhang, Y., Jiang, N., Zeng, T., Hou, C., et al. (2021) · Aging · doi:10.18632/aging.202605 · Cited by 22
Adamse, I., Eichelsheim, V., Blokland, A., & Schoonmade, L. (2024) · European Journal of Criminology · doi:10.1177/14773708241250278 · Cited by 20
“The quality of the included studies was assessed by means of a coding protocol, based on an adaptation of the PROBAST risk-of-bias tool for prediction models (Moons et al., 2019).”
Malik, S., Tenorio, B. G., Moond, V., Dahiya, D. S., Vora, R. S., & Dbouk, N. (2024) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.16645 · Cited by 27
“17 Statistical analyses.”
Grubic, N., Johnston, A., Randhawa, V. K., Humphries, K. H., Rosella, L. C., & Maximova, K. (2024) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2024.12.022 · Cited by 35
Williams, E., Huynh, D., Estai, M., Sinha, T., Summerscales, M., & Kanagasingam, Y. (2025) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2025.100197 · Cited by 27
“5,11,12 Model performance was determined by reported area under the receiver operator characteristic curve (AUROC).”
Prediction models for self-harm and suicide: a systematic review and critical appraisal
Seyedsalehi, A., Bailey, J. P. M., Ogonah, M. G. T., Fanshawe, T. R., & Fazel, S. (2025) · BMC Medicine · doi:10.1186/s12916-025-04367-6 · Cited by 29
“A standardised data extraction and risk of bias assessment form was developed based on the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Model-ling Studies (CHARMS) [27] and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) [28, 29].”
InfluentialPrognostic models for the clinical management of malaria and its complications: a systematic review
Njim, T. N., & Tanyitiku, B. S. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030793 · Cited by 22
Risk prediction models for acute kidney injury in adults: An overview of systematic reviews
Acker, P. V., Biesen, W. V., Nagler, E. V., Koobasi, M., Veys, N., & Vanmassenhove, J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0248899 · Cited by 20
Shi, L., Zhao, J., Peng, X., Wang, Y., Liu, L., & Sheng, M. (2021) · European Journal of Radiology · doi:10.1016/j.ejrad.2021.109956 · Cited by 22
Zou, H., Ge, Y., Lei, Q., Ung, C. O. L., Ruan, Z., Lai, Y., Yao, D., & Hu, H. (2022) · Hepatology International · doi:10.1007/s12072-021-10286-4 · Cited by 20
Senent‐Valero, M., Librero, J., & Pastor‐Valero, M. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01856-6 · Cited by 23
Munguía‐Realpozo, P., Etchegaray‐Morales, I., Pinto, C. M., Méndez‐Martínez, S., Osorio-Peña, Á. D., Ayón-Aguilar, J., & García‐Carrasco, M. (2023) · Autoimmunity Reviews · doi:10.1016/j.autrev.2023.103294 · Cited by 24
Prediction model for cognitive frailty in older adults: A systematic review and critical appraisal
Huang, J., Zeng, X., Hu, M., Ning, H., Wu, S., Peng, R., & Feng, H. (2023) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2023.1119194 · Cited by 21
“Two reviews (JH and XZ) independently assessed the ROB and application of included studies using the Prediction Model Risk of Bias Assessment Tool (PROBAST) (Moons et al., 2019; Wolff et al., 2019).”
MethodInfluentialMachine learning prediction models for diabetic kidney disease: systematic review and meta-analysis
Chen, L., Shao, X., & Yu, P. (2023) · Endocrine · doi:10.1007/s12020-023-03637-8 · Cited by 29
“The risk of bias and concern for applicability was assessed with the Prediction Model Risk of Bias Assessment Tool (PRO-BAST), which was speci fi cally developed to assess diagnostic and prognostic prediction models [17].”
van Hal, S. T., van der Jagt, M., van Genderen, M. E., Gommers, D. A. M. P. J., & Veenland, J. F. (2024) · Neurocritical Care · doi:10.1007/s12028-023-01910-2 · Cited by 27
“Data samples were taken every 0.008 s [17] up to one value per 24 h [10].”
BackgroundMethodResultInfluentialAstley, J., Reilly, J. M., Robinson, S., Wild, J. M., Hatton, M. Q. F., & Tahir, B. (2024) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2024.110084 · Cited by 21
Perkins, Z., Yet, B., Marsden, M. E. R., Glasgow, S., Marsh, W., Davenport, R. A., Brohi, K., & Tai, N. R. M. (2021) · PubMed · doi:10.1097/sla.0000000000003771 · Cited by 19
van Eekhout, J. C. A., Becking, E. C., Scheffer, P. G., Koutsoliakos, I., Bax, C. J., Henneman, L., Bekker, M. N., & Schuit, E. (2024) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17983 · Cited by 27
Hu, Q., Li, J., Li, X., Zou, D., Xu, T., & He, Z. (2024) · Journal of International Medical Research · doi:10.1177/03000605241302304 · Cited by 31
“23 The quality of artificial intelligence studies in the medical field was assessed by implementing the Checklist for the Assessment of Medical AI (ChAMAI).”
Arshi, B., Cowley, L., Rijnhart, E., Reeve, K. A., Smits, L. J. M., & Wynants, L. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111902 · Cited by 34
Preece, R., Dilaver, N., Waldron, C.-A., Pallmann, P., Thomas‐Jones, E., Gwilym, B. L., Norvell, D. C., Czerniecki, J. M., Twine, C. P., & Bosanquet, D. C. (2021) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2021.02.038 · Cited by 21
Monahan, A. C., & Feldman, S. S. (2021) · JMIR Medical Informatics · doi:10.2196/30022 · Cited by 23
Poorthuis, M. H. F., Herings, R. A. R., Dansey, K. D., Damen, J. A., Greving, J. P., Schermerhorn, M. L., & de Borst, G. J. (2021) · Stroke · doi:10.1161/strokeaha.120.032527 · Cited by 20
D'Souza, R. D., Ashraf, R., & Foroutan, F. (2021) · Best Practice & Research Clinical Obstetrics & Gynaecology · doi:10.1016/j.bpobgyn.2021.12.005 · Cited by 19
Klunder, J. H., Panneman, S. L., Wallace, E., de Vries, R., Joling, K. J., Maarsingh, O. R., & van Hout, H. P. J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0275116 · Cited by 23
Risk prediction models for breast cancer-related lymphedema: A systematic review and meta-analysis
Shen, A., Wei, X., Zhu, F., Sun, M., Ke, S., Qiang, W., & Lü, Q. (2023) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2023.102326 · Cited by 23
Langenhuijsen, L. F. S., Janse, R. J., Venema, E., Kent, D. M., van Diepen, M., Dekker, F. W., Steyerberg, E. W., & de Jong, Y. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.04.012 · Cited by 21
A scoring tool to predict mortality and dependency after cerebral venous thrombosis
Lindgren, E., Krzywicka, K., de Winter, M. A., van Kammen, M. S., Heldner, M. R., Hiltunen, S., de Sousa, D. A., Mansour, M., Canhão, P., Ekizoğlu, E., et al. (2023) · European Journal of Neurology · doi:10.1111/ene.15844 · Cited by 20
“To evade data- driven bias, thresholds should be based on clinical relevance [15].”
MethodGinex, P., Dickman, E., Elia, M. R., Burbage, D., Wilson, R., Koos, J. A., Sivakumaran, K., & Morgan, R. L. (2023) · Supportive Care in Cancer · doi:10.1007/s00520-023-07842-z · Cited by 23
Vagliano, I., Dormosh, N., Rios, M., Luik, T. T., Buonocore, T. M., Elbers, P., Dongelmans, D. A., Schut, M. C., & Abu–Hanna, A. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104504 · Cited by 22
Validity of Clinical Severity Scores for Respiratory Syncytial Virus: A Systematic Review
Sheikh, Z., Potter, E., Li, Y., Cohen, R., Santos, G. D., Bont, L., Nair, H., PROMISE Investigators, Nair, H., Campbell, H., et al. (2023) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiad436 · Cited by 23
What’s new in dementia risk prediction modelling? An updated systematic review
Brain, J., Kafadar, A. H., Errington, L., Kirkley, R., Tang, E. Y. H., Akyea, R. K., Bains, M., Brayne, C., Figueredo, G. P., Greene, L., et al. (2024) · Dementia and Geriatric Cognitive Disorders Extra · doi:10.1159/000539744 · Cited by 38
“As such, calibration should also be evaluated using calibration plots or tables comparing predicted versus observed outcome frequencies, as the Hosmer-Lemeshow test alone has limited suitability for assessing calibration, particularly in large datasets [118].”
MethodHando, B., Scott, W. C., Bryant, J. F., Tchandja, J. N., & Angadi, S. S. (2022) · The American Journal of Sports Medicine · doi:10.1177/03635465221083672 · Cited by 19
Liu, X., Liu, X., Jin, C., Luo, Y., Yang, L., Ning, X., Zhuo, C., & Xiao, F. (2024) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2024.07.005 · Cited by 23
Angelantonio, E. D., Pennells, L. A. M., Abdelhamid, M., Aboyans, V., Asteggiano, R., Čelutkienė, J., Grobbee, D. E., Iung, B., Jüni, P., McEvoy, J. W., et al. (2025) · European Heart Journal · doi:10.1093/eurheartj/ehaf016 · Cited by 21
Silva, M. A., Hamilton, E. J., Russell, D., Game, F. L., Wang, S. C., Baptista, S., & Monteiro‐Soares, M. (2025) · Journal of Medical Internet Research · doi:10.2196/69408 · Cited by 28
““EPV” refers to the number of events (ie, number of patients in which the outcome of interest has occurred) relative to the number of regression coefficients used (ie, number of variables considered for model development) [27].”
In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review
Jiménez-Sánchez, A., Avlona, N.-R., de Boer, S., Campello, V. M., Feragen, A., Ferrante, E., Ganz, M., Gichoya, J. W., González, C., Groefsema, S., et al. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732035 · Cited by 36
“We briefly discuss considerations about the trade-off between metadata and patient privacy in Section 3, and considerations about FAIR principles and licensing in Section 7.”
Cooray, S. D., Boyle, J. A., Soldatos, G., Wijeyaratne, L. A., & Teede, H. J. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1151-0 · Cited by 21
Risk prediction models for emergence delirium in paediatric general anaesthesia: a systematic review
Petre, M.-A., Saha, B., Kasuya, S., Englesakis, M. F., Gai, N., Peliowski, A., & Aoyama, K. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-043968 · Cited by 18
Ahmadian, S., Sin, D. D., Lynd, L. D., Harrison, M. J., & Sadatsafavi, M. (2021) · Thorax · doi:10.1136/thoraxjnl-2021-217962 · Cited by 20
Haller, M. C., Aschauer, C., Wallisch, C., Leffondré, K., van Smeden, M., Oberbauer, R. W., & Heinze, G. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.01.025 · Cited by 21
Tohidinezhad, F., Perri, D. D., Zegers, C. M. L., Dijkstra, J. B., Anten, M. H. M. E., Dekker, A. L. A. J., van Elmpt, W., Eekers, D. B. P., & Traverso, A. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.853472 · Cited by 18
Pandor, A., Daru, J., Hunt, B. J., Rooney, G., Hamilton, J., Clowes, M., Goodacre, S. W., Nelson‐Piercy, C., & Davis, S. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-065892 · Cited by 21
Metadata
- Authors
- Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S.
- Affiliations
- Utrecht University (NL)
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- W1-W33
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907638671
- Cited by
- 1,669
- References
- 201
- Citations by year
- 2026: 359 · 2025: 330 · 2024: 253 · 2023: 236 · 2022: 193 · 2021: 175 · 2020: 96 · 2019: 24
- FWCI
- 56.12
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (publishedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- PROBAST, risk of bias assessment, applicability, prediction model, diagnostic model, prognostic model, model validation
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.