PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration
Open accessThe DOI matches its Crossref record; 194 of the 201 references it cites were found in a registry or an index.
Abstract
Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMeasuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Research works printed but not linked: 2
- 10.1177/0962280218785504.30032705 · doi:10.1177/0962280218785504.30032705
- Wong SS, 2003, AMIA Annu Symp Proc, 728
Cited by
Items 501–600 of 1,003
Whiting, D., Mallett, S., Lennox, B., & Fazel, S. (2023) · BMJ Mental Health · doi:10.1136/bmjment-2022-300634 · Cited by 14
“Alternatives such as complete-case analysis are recognised as unnecessarily inefficient as they can disregard a substantial amount of available data and introduce unwanted selection bias because only patients with complete data can be included in analysis [3].”
BackgroundSelf-citationLi, J., Chen, Y., Chen, S., Wang, S., Zhang, D., Wang, J., Postmus, D., Zeng, H., Qin, G., Shen, Y., et al. (2020) · BMC Infectious Diseases · doi:10.1186/s12879-020-05688-y · Cited by 26
Jiang, Y., Li, A., Li, Z., Li, Y., Li, R., Zhao, Q., & Li, G. (2025) · PLoS ONE · doi:10.1371/journal.pone.0302676 · Cited by 19
“Model performance was evaluated using area under the receiver operating characteristic curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).”
Hunik, L., Chaabouni, A., van Laarhoven, T., Hartman, T. O., Leijenaar, R. T. H., Cals, J., Uijen, A. A., & Schers, H. (2025) · JMIR Medical Informatics · doi:10.2196/62862 · Cited by 21
Diagnostic and prognostic factors in patients with prostate cancer: a systematic review
Beyer, K., Moris, L., Lardas, M., Haire, A., Barletta, F. M., Scuderi, S., Molnar, M., Herrera, R., Rauf, A., Campi, R., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058267 · Cited by 14
Klén, R., Purohit, D., Gómez‐Huelgas, R., Rojo, J. M. C., Santos, J. M. A., Núñez-Cortés, J. M., Lumbreras, C., Ramos, J. M., Barrio, N. G., Pedrera‐Jiménez, M., et al. (2021) · eLife · doi:10.2139/ssrn.3920914 · Cited by 16
van de Kuit, A., Oosterhoff, J. H. F., Dijkstra, H., Sprague, S., Bzovsky, S., Bhandari, M., Swiontkowski, M. F., Schemitsch, E. H., IJpma, F. F. A., Poolman, R. W., et al. (2022) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000002283 · Cited by 14
Adeoye, J., Akinshipo, A., Koohi‐Moghadam, M., Thomson, P. J., & Su, Y. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.976168 · Cited by 15
“The suboptimal models in this review were found to be a culmination of one of the following challenges: model overfitting, reduced cohort size, not incorporating methods to augment data or correct outcome imbalance, lack of model stability assessment, and cessation of further model validity assessment using external c…”
MethodDhiman, P., Ma, J., Gibbs, V. N., Rampotas, A., Kamal, H., Arshad, S., Kirtley, S., Doree, C. J., Murphy, M. F., Collins, G. S., et al. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.05.002 · Cited by 17
Gokhale, S., Taylor, D., Gill, J., Hu, Y., Zeps, N., Lequertier, V., Teede, H. J., & Enticott, J. (2023) · Digital Health · doi:10.1177/20552076231177497 · Cited by 16
“and/or external validation) was based on PROBAST guideline.(28)”
MethodWainstein, M., Flanagan, E., Johnson, D. W., & Shrapnel, S. (2023) · Frontiers in Nephrology · doi:10.3389/fneph.2023.1220214 · Cited by 16
“The latter results in incorporation bias whereby the association between the predictor and the outcome is overestimated rendering model performance evaluations optimistic (11).”
BackgroundMethodFeng, J., Zhang, Q., Wu, F., Peng, J., Li, Z., & Chen, Z. (2023) · Journal of Medical Internet Research · doi:10.2196/44895 · Cited by 12
“The PROBAST (Prediction Model Risk of Bias Assessment Tool) was used [22] to evaluate the risk of bias in the included studies.”
Li, Y., Zhang, R., Dong, L., Shi, X., Zhou, W., Wu, H., Li, H., Yu, C., & Wei, W. (2024) · Eye · doi:10.1038/s41433-023-02914-0 · Cited by 14
“Based on PROBAST (prediction model risk of bias assessment tool) [23, 24], we used a modified version that recorded items except for applicability and predictor variables, the same as the items used in a methodology review published previously [21].”
Yoong, S. Q., Bhowmik, P., Kapparath, S., & Porock, D. C. (2024) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djae036 · Cited by 14
Li, J., Li, G., Liu, Z., Yang, X., & Yang, Q. (2024) · American Journal of Infection Control · doi:10.1016/j.ajic.2024.07.006 · Cited by 13
Sun, J., Feng, C., Liu, Y., Shan, M., Wang, Z., Fu, W., & Niu, W. (2024) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2024.1435807 · Cited by 12
Walker, H., Day, S., Grant, C. H., Jones, C., Ker, R., Sullivan, M. K., Jani, B., Gallacher, K., & Mark, P. B. (2024) · BMC Medicine · doi:10.1186/s12916-024-03649-9 · Cited by 13
“Two authors, HW and a second reviewer (SD, CHG, CJ or RK), independently reviewed the risk of bias using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) [29].”
MethodBoldrini, L., Charles-Davies, D., Romano, A., Mancino, M., Nacci, I., Tran, H. E., Bono, F., Boccia, E., Gambacorta, M. A., & Chiloiro, G. (2024) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2024.109463 · Cited by 13
Abdaltawab, A., Chang, L., Mansour, M., & Koubeissi, M. Z. (2024) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2024.110212 · Cited by 16
Meinke, C., Hornstein, S., Schmidt, J., Arolt, V., Dannlowski, U., Deckert, J., Domschke, K., Fehm, L., Fydrich, T., Gerlach, A. L., et al. (2024) · Psychological Medicine · doi:10.1017/s0033291724003118 · Cited by 14
Prediction models for treatment response in migraine: a systematic review and meta-analysis
Chen, Q., Zhang, J., Cao, B., Hu, Y., Kong, Y., Li, B., & Liu, L. (2025) · The Journal of Headache and Pain · doi:10.1186/s10194-025-01972-x · Cited by 15
“A robust sample size is critical for maintaining the events-per-variable ratio, which prevents overfitting and enhances model reliability [29] .”
InfluentialBaêta, T., Rocha, A. L. L., Oliveira, J. A., da Silva, A. P. C., & Reis, Z. S. N. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-094455 · Cited by 18
Utsumi, T., Ishitsuka, N., Noro, T., Suzuki, Y., Iijima, S., Sugizaki, Y., Somoto, T., Oka, R., Endo, T., Kamiya, N., et al. (2025) · International Journal of Urology · doi:10.1111/iju.70075 · Cited by 19
Tong, D., Midroni, J., Avison, K., Alnassar, S., Chen, D. Y., Parsa, R., Yariv, O., Liu, Z., Ye, X. Y., Hope, A., et al. (2025) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2025.110935 · Cited by 12
Kuo, K.-M., & Chang, C. S. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03010-x · Cited by 21
“This study assessed the risk of bias and applicability of the evidence based on the Prediction model risk of bias assessment tool (PROBAST) [106, 107].”
Zhang, H., Chen, D., Shao, J., Zou, P., Cui, N., Tang, L., Wang, D., & Ye, Z. (2021) · Diabetes Metabolic Syndrome and Obesity · doi:10.2147/dmso.s288881 · Cited by 13
Beaulieu, D., Berry, J. D., Paganoni, S., Glass, J. D., Fournier, C. N., Cuerdo, J., Schactman, M., & Ennist, D. L. (2021) · Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration · doi:10.1080/21678421.2021.1924207 · Cited by 15
Belkouz, A., van Roessel, S. R., Strijker, M., van Dam, J. L., Daamen, L. A., van der Geest, L. G., Balduzzi, A., Cacciaguerra, A. B., van Dieren, S., Molenaar, I. Q., et al. (2022) · British Journal of Cancer · doi:10.1038/s41416-021-01687-1 · Cited by 12
Kaiser, I., Mathes, S., Pfahlberg, A. B., Uter, W. J. C., Berking, C., Heppt, M. V., Steeb, T., Diehl, K., & Gefeller, O. (2022) · Cancers · doi:10.3390/cancers14123033 · Cited by 14
Majithia-Beet, G., Naemi, R., & Issitt, R. W. (2022) · Perfusion · doi:10.1177/02676591221115267 · Cited by 12
Gehringer, C. K., Martin, G. P., Hyrich, K. L., Verstappen, S. M. M., & Sergeant, J. C. (2022) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2022.152076 · Cited by 14
Rademaker, M. M., Meijers, S. M., Smit, A. L., & Stegeman, I. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12020695 · Cited by 13
Yuan, E., Chen, Y., & Song, B. (2023) · European Radiology · doi:10.1007/s00330-023-09414-5 · Cited by 13
“However, this method has been shown to be inefficient with small sample size [46].”
MethodArshi, B., Wynants, L., Rijnhart, E., Reeve, K. A., Cowley, L., & Smits, L. J. M. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073174 · Cited by 16
Factors associated with suicidal ideation in patients with multiple myeloma
Akechi, T., Okuyama, T., Uchida, M., Kubota, Y., Hasegawa, T., Suzuki, N., Komatsu, H., Kusumoto, S., & Iida, S. (2020) · Japanese Journal of Clinical Oncology · doi:10.1093/jjco/hyaa143 · Cited by 14
Asiimwe, I. G., Waitt, C. J., Sekaggya‐Wiltshire, C., Hutchinson, C., Okello, E., Zhang, E. J., Semakula, J. R., Mouton, J. P., Cohen, K., Blockman, M., et al. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.2128 · Cited by 12
Yan, A., Samarawickrema, I., Naunton, M., Peterson, G. M., Yip, D., Rosa, S. D., & Mortazavi, R. (2021) · Healthcare · doi:10.3390/healthcare9060778 · Cited by 13
Simon-Pimmel, J., Foucher, Y., Léger, M., Feuillet, F., Bodet-Contentin, L., Cinotti, R., Frasca, D., & Dantan, É. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047279 · Cited by 13
Chen, H., Poon, I., Atenafu, E. G., Badellino, S., Biswas, T., Dagan, R., Erler, D., Foote, M. C., Redmond, K. J., Ricardi, U., et al. (2021) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2021.12.001 · Cited by 12
Prediction models in gynaecology: Transparent reporting needed for clinical application
Gelderblom, M. E., Stevens, K. Y. R., Houterman, S., Weyers, S., & Schoot, B. C. (2021) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2021.08.013 · Cited by 11
Wang, B., Jiang, B., Liu, D., & Zhu, R. (2025) · Journal of Medical Internet Research · doi:10.2196/71654 · Cited by 20
Yang, W., Wong, H. J., Fu, C. E., Rojas-Carabali, W. A., Agrawal, R. V., & Ang, B. C. H. (2025) · Survey of Ophthalmology · doi:10.1016/j.survophthal.2025.06.006 · Cited by 15
Susanty, S., Sufriyana, H., Su, E. C., & Chuang, Y. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280330 · Cited by 14
“A sufficient sample size was also considered according to the PROBAST guidelines since a small sample size was vulnerable to overfitting [58].”
BackgroundMethodInfluentialOtotoxicity prognostic models in adult and pediatric cancer patients: a rapid review
Debacker, J. R., McMillan, G. P., Martchenke, N., Lacey, C., Stuehm, H. R., Hungerford, M. E., & Konrad‐Martin, D. (2023) · Journal of Cancer Survivorship · doi:10.1007/s11764-022-01315-8 · Cited by 12
“5 (column 22 of online supplemental table) asks, “Was selection of predictors based on univariable analysis avoided?” As Moons and colleagues [41] explain, predictors should not be chosen based on their independent statistical significance but rather on their influence on the outcome in conjunction with other predicto…”
BackgroundMethodAly, F., Hansen, C. R., Mouiee, D. A., Sundaresan, P., Haidar, A., Vinod, S. K., & Holloway, L. C. (2023) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2023.109629 · Cited by 11
Performance of a Prediabetes Risk Prediction Model: A Systematic Review
Liu, Y., Feng, W., Lou, J., Qiu, W., Shen, J., Zhu, Z., Hua, Y., Zhang, M., & Billong, L. F. (2022) · Heliyon · doi:10.1016/j.heliyon.2023.e15529 · Cited by 18
“Since 2 studies [31,32] used classical machine learning and deep learning techniques and could not be included in the PROBAST quality assessment, the remaining 12 studies [17,20–30] were assessed for risk of bias.”
MethodInfluentialHerrera, D. J., van de Veerdonk, W., Seibert, D. M., Boke, M. M., Gutiérrez‐Ortiz, C., Yimer, N. B., Feyen, K., Ferrari, A., & Hal, G. F. V. (2023) · Gastrointestinal Disorders · doi:10.3390/gidisord5040045 · Cited by 14
“The PROBAST tool enables a focused and transparent approach to assess the RoB and applicability of studies that develop, validate, or update prediction models for individualized predictions [27] (Figure 1).”
Lang, F.-F., Liu, L., & Wang, S. (2023) · Frontiers in Physiology · doi:10.3389/fphys.2023.1306453 · Cited by 13
Yan, A., Samarawickrema, I., Naunton, M., Peterson, G. M., Yip, D., Newman, P., & Mortazavi, R. (2024) · Thrombosis Research · doi:10.1016/j.thromres.2024.01.003 · Cited by 12
Rayner, D. G., Kim, B. Y., & Foroutan, F. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111360 · Cited by 16
Clinical prediction models for treatment outcomes in newly diagnosed epilepsy: A systematic review
Ratcliffe, C., Pradeep, V., Marson, A. G., Keller, S. S., & Bonnett, L. J. (2024) · Epilepsia · doi:10.1111/epi.17994 · Cited by 12
“36 To best meet our primary objective—to inform the prognostic factor choices of future prediction model studies—this report has been prepared in accordance with PRISMA (where appropriate), ensuring maximum transparency, reproducibility, and clarity.”
InfluentialJensen, M. P., Qiang, Z., Khan, D. Z., Stoyanov, D. V., Baldeweg, S. E., Jaunmuktane, Z., Brandner, S., & Marcus, H. J. (2024) · Neuropathology and Applied Neurobiology · doi:10.1111/nan.12981 · Cited by 17
“14 A narrative synthesis was conducted to provide a comprehensive summary of the study characteristics, AI techniques employed, and key findings.”
MethodThe prognostic value of radiogenomics using CT in patients with lung cancer: a systematic review
Jiang, Y., Gao, C., Shao, Y., Lou, X., Hua, M., Lin, J., Wu, L., & Gao, C. (2024) · Insights into Imaging · doi:10.1186/s13244-024-01831-4 · Cited by 17
“Bias risk assessment assessed the methodological quality of each study using a radiomics quality score (RQS) and the predictive model bias risk assessment tool (PROBAST) [24, 25].”
MethodDevelopment and Validation of Dynamic 5-Year Breast Cancer Risk Model Using Repeated Mammograms
Jiang, S., Bennett, D. L., ROSNER, B. A., Tamimi, R. M., & Colditz, G. A. (2024) · JCO Clinical Cancer Informatics · doi:10.1200/cci-24-00200 · Cited by 15
Risk Prediction Models as an Emerging Trend for Managing Cancer‐Related Fatigue: A Systematic Review
Zhang, Y., Li, L., Li, X., Zhang, S., Zhou, L., Chen, X., & Hu, X. (2024) · Journal of Advanced Nursing · doi:10.1111/jan.16680 · Cited by 12
Mortality Risk Prediction Models for People With Kidney Failure: A Systematic Review
Jarrar, F., Pasternak, M., Harrison, T. G., James, M. T., Quinn, R. R., Lam, N. N., Donald, M., Elliott, M. J., Lorenzetti, D., Strippoli, G. F., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.53190 · Cited by 18
Stephan, A., Hanselmann, M., Bajramovic, M., Schoßer, S., & Laxy, M. (2025) · Cardiovascular Diabetology · doi:10.1186/s12933-025-02640-9 · Cited by 14
“We pre-specified and published model development plans [10, 11] for both project parts before carrying out the analyses, considering potential sources of bias outlined in the Prediction model Risk Of Bias ASsessment Tool (PROBAST) [12].”
InfluentialTao, H., Hui, X., Zhang, Z. H., Zhu, R., Wang, P., Zhou, S., & Yang, K. (2025) · BMC Cancer · doi:10.1186/s12885-025-13631-0 · Cited by 12
Federated Learning-Based Model for Predicting Mortality: Systematic Review and Meta-Analysis
Tahir, N., Jung, C., Lee, S., Azizah, N., Ho, W., & Li, T. (2025) · Journal of Medical Internet Research · doi:10.2196/65708 · Cited by 19
Risk Prediction Models for Perioperative Hypothermia: A Systematic Review
Liu, J., Liu, F., Xu, W., Du, L., Li, Y., Liang, A., Li, B., & Zhang, M. (2025) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s538891 · Cited by 16
“10,11 Risk prediction models are widely applied across medical practice for multiple purposes, including predicting disease development, treatment response, or patient prognosis.”
Brada, L. J. H., Walma, M. S., Daamen, L. A., van Roessel, S. R., van Dam, R. M., de Hingh, I. H. J. T., Liem, M. S. L., de Meijer, V. E., Patijn, G. A., Festen, S., et al. (2021) · Journal of Surgical Oncology · doi:10.1002/jso.26567 · Cited by 11
Soeteman, M., Lekkerkerker, C. W. M., Kappen, T. H., Tissing, W. J. E., Nieuwenhuis, E. E. S., & Asperen, R. M. W. (2022) · Pediatric Blood & Cancer · doi:10.1002/pbc.29636 · Cited by 13
Elwenspoek, M. M. C., O’Donnell, R., Jackson, J., Everitt, H., Gillett, P. M., Hay, A. D., Jones, H. E., Robins, G. G., Watson, J., Mallett, S., et al. (2022) · EClinicalMedicine · doi:10.1016/j.eclinm.2022.101376 · Cited by 13
Self-citationHuang, Q., Hu, Y.-C., Wang, C.-K., Huang, J., Shen, M., & Ren, L. (2022) · Biological Research For Nursing · doi:10.1177/10998004221131993 · Cited by 17
Araújo, A. L. D., Santos‐Silva, A. R., & Kowalski, L. P. (2023) · Current Oncology Reports · doi:10.1007/s11912-023-01365-w · Cited by 13
Kwok, S. W. H., Wang, G., Sohel, F. A., KASHANI, K. B., Zhu, Y., Wang, Z., Antpack, E., Khandelwal, K., Pagali, S. R., Nanda, S., et al. (2023) · Respiratory Research · doi:10.1186/s12931-023-02386-6 · Cited by 12
“Although previously identified ML models achieved modest to excellent discriminatory performance, the studies were at high-risk of bias when assessed using “Prediction model Risk Of Bias Assessment Tool (PROBAST)” [74] [4, 73].”
MethodAbesi, F., & Golikani, A. (2023) · Polish Journal of Radiology · doi:10.5114/pjr.2023.134035 · Cited by 11
“We utilized the adapted Prediction Model Risk of Bias Assessment Tool (PROBAST) criteria to assess the risk of bias of the included studies [13].”
MethodEissa, A., Elsherbiny, A., Zoeir, A., Sandri, M. F., Pirola, G. M., Puliatti, S., Prete, C. D., Sighinolfi, M. C., Micali, S., Rocco, B., et al. (2019) · Minerva Urologica e Nefrologica · doi:10.23736/s0393-2249.19.03427-1 · Cited by 12
Gates, M., Pillay, J., Thériault, G., Limburg, H., Grad, R. M., Klarenbach, S. W., Korownyk, C. S., Reynolds, D. L., Riva, J. J., Thombs, B. D., et al. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1094-5 · Cited by 13
Xu, W., Huang, J., Yu, Q., Yu, H., Pu, Y., & Shi, Q. (2021) · Neurological Sciences · doi:10.1007/s10072-021-05219-w · Cited by 12
Wingbermühle, R. W., Chiarotto, A., van Trijffel, E., Koes, B. W., Verhagen, A. P., & Heymans, M. W. (2021) · Physiotherapy · doi:10.1016/j.physio.2021.05.011 · Cited by 14
Harrison, H., Pennells, L. A. M., Wood, A., Rossi, S. H., Stewart, G. D., Griffin, S. J., & Usher‐Smith, J. A. (2021) · British Journal of Urology · doi:10.1111/bju.15598 · Cited by 12
Systematic review of prediction models for postacute care destination decision-making
Kennedy, E. E., Bowles, K. H., & Aryal, S. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab197 · Cited by 13
Wang, Y., Zheng, R., Wu, Y., Liu, T., Hao, L., Liu, J., Shi, L., & Guo, Q. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105094 · Cited by 16
Brinza, C., Popa, I. V., Basarab, A., Crișan-Dabija, R. A., & Burlacu, A. (2021) · Healthcare · doi:10.3390/healthcare9081033 · Cited by 10
Nguyên, Q. D., Moodie, E. E. M., Desmarais, P., Goulden, R., Forget, M., Peters, E., Saeed, S., Keezer, M. R., & Wolfson, C. M. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01445-0 · Cited by 14
Nomali, M., Khalili, D., Yaseri, M., Mansournia, M. A., Ayati, A., Navid, H., & Nedjat, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0292396 · Cited by 12
“To assess the risk of bias in included studies, we used the prediction model risk of bias assessment tool (PROBAST) [30].”
Seow‐En, I., Koh, Y. X., Zhao, Y. J., Ang, B. H., Tan, I. E., Chok, A. Y., Tan, E., & Au, M. K. H. (2023) · Annals of Hepato-Biliary-Pancreatic Surgery · doi:10.14701/ahbps.23-078 · Cited by 13
“The risk of bias (RoB) of the selected studies was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST) [20].”
Contesting the conventional wisdom of periodontal risk assessment
Raittio, E. J., López, R., & Bælum, V. (2024) · Community Dentistry And Oral Epidemiology · doi:10.1111/cdoe.12942 · Cited by 11
Byrne, J. F., Healy, C., Föcking, M., Susai, S. R., Mongan, D., Wynne, K. J., Kodosaki, E., Heurich, M., de Haan, L., Hickie, I. B., et al. (2024) · Schizophrenia Bulletin · doi:10.1093/schbul/sbad184 · Cited by 11
“40,41 This investigation ensured that there was a large enough sample size to fit 10 predictors in a model, based on recommended sample size calculations.”
Sun, Y., Liu, Y., Zhu, Y., Luo, R., Luo, Y., Wang, S., & Feng, Z. (2024) · Current Medical Research and Opinion · doi:10.1080/03007995.2024.2307346 · Cited by 14
“In this investigation, the Prediction Model Risk of Bias Assessment Tool (PROBAST) was employed to systematically evaluate the potential bias and applicability of multifactorial prediction models 23,24 .”
MethodHorner, D. E., Davis, S., Pandor, A., Shulver, H., Goodacre, S. W., Hind, D., Rex, S., Gillett, M., Bursnall, M., Griffin, X. L., et al. (2024) · Health Technology Assessment · doi:10.3310/awtw6200 · Cited by 10
Haris, M., Raveendra, K., Travlos, C. K., Lewington, A. J. P., Wu, J., Shuweihdi, F., Nadarajah, R., & Gale, C. P. (2024) · Clinical Kidney Journal · doi:10.1093/ckj/sfae098 · Cited by 14
“Three investigators ( M.H., K.R. and C.K.T.) assessed the models for risk of bias and applicability to our review question using the Prediction model Risk Of Bias ASsessment Tool ( PROBAST) [25 ].”
MethodZhang, L., Jin, S., Wang, Y., Zhang, Z., Jia, H., Li, D., & Lü, Q. (2024) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2024.110339 · Cited by 10
Lokhorst, C., van der Werf, S., Berger, R. M. F., & Douwes, J. M. (2024) · Journal of the American Heart Association · doi:10.1161/jaha.123.034151 · Cited by 12
“18 The third approach was either selecting the last available fol- low-up data or the follow-up moment with most available variables, imputing missing variables, or allowing missing variables within the risk stratification model, and there was no need to exclude patients or impute data.”
BackgroundInfluentialPerformance of risk prediction models for diabetic foot ulcer: a meta-analysis
Guo, P., Tu, Y., Liu, R., Gao, Z., Du, M., Fu, Y., Wang, Y., Yan, S., & Shang, X. C. (2024) · PeerJ · doi:10.7717/peerj.17770 · Cited by 16
Risk prediction model for postoperative pneumonia in esophageal cancer patients: A systematic review
Jiang, Y., Li, Z., Jiang, W., Wei, T., & Chen, B. (2024) · Frontiers in Oncology · doi:10.3389/fonc.2024.1419633 · Cited by 14
Han, M., Zhao, Q., Zhao, J., Xue, X., & Wu, H. (2024) · World Journal of Surgery · doi:10.1002/wjs.12335 · Cited by 10
Cox, E. G. M., Meijs, D. A. M., Wynants, L., Sels, J.-W. E. M., Koeze, J., Keus, F., Dongen, B. B.-V., van der Horst, I. C. C., & van der Horst, I. C. C. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111605 · Cited by 12
Crawford, K., Carlo, W. A., Odibo, A. O., Papageorghiou, A. T., Tarnow-Mordi, W., & Kumar, S. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103076 · Cited by 11
Yismaw, M. B., Tafere, C., Tefera, B. B., Demsie, D. G., Feyisa, K., Addisu, Z. D., Zeleke, T. K., Siraj, E. A., Worku, M. C., & Berihun, F. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105858 · Cited by 11
Yang, J., Zeng, S., Cui, S., Zheng, J., & Wang, H. (2025) · Journal of Medical Internet Research · doi:10.2196/66615 · Cited by 13
“PROBAST consists of 4 domains (study population, predictors, outcomes, and statistical analyses), comprising a total of 20 questions [14].”
Aant, N., Arabbeiki, M., & Niroomand, M. R. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110744 · Cited by 10
Moro, F., Momi, M., Ledger, A., Barreñada, L., Ceusters, J., Sturla, D., Mor, E., Fornari, L., Mascilini, F., Ciccarone, F., et al. (2025) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2025.07.017 · Cited by 13
Machine Learning for Warfarin Therapy: A Systematic Review
Fülöp, P., Tóth, Š., Porubän, T., Fülöpová, Z., Borovská, A., & Dvorožňáková, M. (2025) · Pharmaceuticals · doi:10.3390/ph18101544 · Cited by 13
“Two independent reviewers (P.F., T.P.) systematically extracted data from each included study using the Prediction Model Risk of Bias Assessment Tool (PROBAST) framework [19].”
Gillman, A., Lunardo, F., Prinable, J., Belous, G., Nicolson, A., Min, H., Terhorst, A., & Dowling, J. (2021) · Physical and Engineering Sciences in Medicine · doi:10.1007/s13246-021-01093-0 · Cited by 12
Liu, F., Chen, D., Zhou, X., Dai, W., & Xu, F. (2022) · Applied Sciences · doi:10.3390/app12083895 · Cited by 12
Sarrió‐Sanz, P., Martínez-Cayuelas, L., Lumbreras, B., Sánchez‐Caballero, L., Palazón‐Bru, A., Gil-Guillén, V. F., & Gómez‐Pérez, L. (2022) · European Journal of Clinical Investigation · doi:10.1111/eci.13822 · Cited by 10
Bérard, E., Röllig, C., Bertoli, S., Pigneux, A., Tavitian, S., Krämer, M., Serve, H. L., Bornhäuser, M., Platzbecker, U., Müller‐Tidow, C., et al. (2022) · Blood Cancer Journal · doi:10.1038/s41408-022-00700-x · Cited by 14
Sun, M., Yan, T., Liu, R., Zhao, X., Zhou, X., Ma, Y., & Jia, J. (2022) · Annals of Palliative Medicine · doi:10.21037/apm-22-1396 · Cited by 13
Metadata
- Authors
- Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S.
- Affiliations
- Utrecht University (NL)
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- W1-W33
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907638671
- Cited by
- 1,669
- References
- 201
- Citations by year
- 2026: 359 · 2025: 330 · 2024: 253 · 2023: 236 · 2022: 193 · 2021: 175 · 2020: 96 · 2019: 24
- FWCI
- 56.12
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (publishedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- PROBAST, risk of bias assessment, applicability, prediction model, diagnostic model, prognostic model, model validation
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.