PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · cited 3×
“A domain-based structure was adopted similar to that used in other risk of bias tools such as the revised Cochrane Risk of Bias tool,(29) QUADAS-2,(36) ROBINSI(30) and ROBIS.”
BackgroundMethodSelf-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · cited 3×
“A domain-based structure was adopted similar to that used in other risk of bias tools such as the revised Cochrane Risk of Bias tool,(29) QUADAS-2,(36) ROBINSI(30) and ROBIS.”
InfluentialSelf-citationCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147 · cited 11×
“A multivariable prediction model is defined as any combination or equation of two or more predictors for estimating the probability or risk for an individual.(7, 8, 33-35)”
InfluentialSelf-citation2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.11.005 · Cited by 4,945
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Circulation · doi:10.1161/01.cir.0000437741.48606.98 · Cited by 3,697
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
Self-citationAssessing Bias in Studies of Prognostic Factors
Hayden, J. A., van der Windt, D. A., Cartwright, J. L., Côté, P., & Bombardier, C. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-4-201302190-00009 · Cited by 3,287 · cited 1×
“(25, 26) The QUIPS tool has been developed to assess the risk of bias in predictor finding (prognostic factor) studies.(28) The methodological quality of studies investigating the impact of a prediction model using a comparative randomised design can be assessed using the revised Cochrane risk of bias tool (ROB 2.”
MethodCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
Self-citationSystematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Cited by
Items 501–600 of 1,010
Language-based detection of depression with machine learning: systematic review and meta-analysis
Fisher, H., Jaffe, N. M., Pidvirny, K., Tierney, A. O., Vaidean, M. S., Dongre, P., & Webb, C. A. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02448-1 · Cited by 29
FINDRISC in Latin America: a systematic review of diagnosis and prognosis models
Carrillo-Larco, R. M., Aparcana-Granda, D. J., Mejia, J. R., & Bernabé‐Ortiz, A. (2020) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2019-001169 · Cited by 27
Reps, J., Kim, C., Williams, R. D., Markus, A. F., Yang, C. B., Duarte‐Salles, T., Falconer, T., Jonnagaddala, J., Williams, A. E., Fernández‐Bertolín, S., et al. (2021) · JMIR Medical Informatics · doi:10.2196/21547 · Cited by 30
“Research has shown that there is high risk of bias for a model that lacks external validation [7].”
Post‐stroke seizure risk prediction models: a systematic review and meta‐analysis
Lee, S. H., Aw, K. L., Banik, S., & Myint, P. K. (2022) · Epileptic Disorders · doi:10.1684/epd.2021.1391 · Cited by 30
Frondelius, T., Atkova, I., Miettunen, J., Rello, J., & Jansson, M. M. (2021) · Journal of Critical Care · doi:10.1016/j.jcrc.2021.10.001 · Cited by 31
Brazzelli, M., Aucott, L. S., Aceves‐Martins, M., Robertson, C., Jacobsen, E., Imamura, M., Poobalan, A. S., Manson, P. D., Scotland, G., Kaye, C. T., et al. (2022) · Health Technology Assessment · doi:10.3310/ugez4120 · Cited by 27
Silva, F. G., Costa, L. O. P., Hancock, M. J., Palomo, G. A., Costa, L. C., & da Silva, T. M. (2022) · Journal of physiotherapy · doi:10.1016/j.jphys.2022.03.009 · Cited by 32
Demont, A., Lafrance, S., Benaissa, L., & Mawet, J. (2022) · Musculoskeletal Science and Practice · doi:10.1016/j.msksp.2022.102640 · Cited by 27
Cox, E. G. M., Wiersema, R., Eck, R. J., Kaufmann, T., Granholm, A., Vaara, S. T., Møller, M. H., van Bussel, B. C. T., Snieder, H., Pleijhuis, R. G., et al. (2022) · Critical Care Medicine · doi:10.1097/ccm.0000000000005712 · Cited by 32
Predictive model for long COVID in children 3 months after a SARS-CoV-2 PCR test
Nugawela, M. D., Stephenson, T. J., Shafran, R., Stavola, B. L. D., Ladhani, S. N., Simmons, R., McOwat, K., Rojas, N. K., Dalrymple, E., Cheung, E. Y., et al. (2022) · BMC Medicine · doi:10.1186/s12916-022-02664-y · Cited by 30
“Based on predictive model quality assessment tools [10] and model development guidelines [11], the two models mentioned above (the Jehi diagnostic model [12] and 4C mortality score [13]) and a third model (QCOVID [14]) are considered as higher quality predictive models for SARS-CoV-2 because of large sample sizes [15]…”
MethodMachine learning with multimodal data for COVID-19
Chen, W., Sá, R. C., Bai, Y., Napel, S. A., Gevaert, O., Lauderdale, D. S., & Giger, M. L. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e17934 · Cited by 31
Munafò, A. R., Montalto, C., Franzino, M., Pistelli, L., Bella, G. D., Ferlini, M., Leonardi, S., D’Ascenzo, F., Gragnano, F., Oreglia, J., et al. (2023) · European Heart Journal - Cardiovascular Pharmacotherapy · doi:10.1093/ehjcvp/pvad063 · Cited by 29
Risk Prediction Models for Oral Cancer: A Systematic Review
Espressivo, A., Pan, Z., Usher‐Smith, J. A., & Harrison, H. (2024) · Cancers · doi:10.3390/cancers16030617 · Cited by 27
“…Prognosis or Diagnosis (TRIPOD) guidelines [30,31] and we assessed both their risk of bias and applicability to a risk-stratified screening programme using the Prediction Model Risk of Bias Assessment Tools (PROBAST) across four domains (population, risk factors, outcomes, and analysis) [32,33].”
MethodData set terminology of deep learning in medicine: a historical review and recommendation
Walston, S. L., Seki, H., Takita, H., Mitsuyama, Y., Sato, S., Hagiwara, A., Ito, R., Hanaoka, S., Miki, Y., & Ueda, D. (2024) · Japanese Journal of Radiology · doi:10.1007/s11604-024-01608-1 · Cited by 29
“While there is some debate on whether temporal test sets should be considered truly external, guidelines such as PROBAST, TRIPOD+AI, and work by Park et al. define them as external test sets [30, 31, 33].”
Fernandez-Felix, B. M., Barca, L. V., García‐Esquinas, E., Correa-Pérez, A., Fernández‐Hidalgo, N., Muriel, A., López‐Alcalde, J., Díaz, N. Á., Pijoán, J. I., Ribera, A., et al. (2021) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2021.05.051 · Cited by 29
Mohammadzadeh, I., Niroomand, B., Eini, P., Khaledian, H., Choubineh, T., & Luzzi, S. (2025) · Neurosurgical Review · doi:10.1007/s10143-024-03175-5 · Cited by 26
“The risk of bias and applicability of included studies were assessed using the PROBAST [20] (Prediction model risk of bias assessment Tool).”
Prognostic models for predicting the severity and mortality in people with acute pancreatitis
Gurusamy, K. S., Debray, T. P. A., & Rompianesi, G. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013026 · Cited by 31
Akyea, R. K., Leonardi‐Bee, J., Asselbergs, F. W., Patel, R. S., Durrington, P. N., Wierzbicki, A. S., Ibiwoye, O. H., Kai, J., Qureshi, N., & Weng, S. F. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-034564 · Cited by 27
Prognostic models for chronic kidney disease: a systematic review and external validation
van Rijn, M. H. C., van de Luijtgaarden, M. W. M., van Zuilen, A. D., Blankestijn, P. J., Wetzels, J. F. M., Debray, T. P. A., & van den Brand, J. A. J. G. (2020) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfaa155 · Cited by 31
Risk Factors for Mobility Decline in Community-Dwelling Older Adults: A Systematic Literature Review
Nicolson, P. J. A., Sanchez-Santos, M. T., Bruce, J., Kirtley, S., Ward, L., Williamson, E. M., & Lamb, S. E. (2021) · Journal of Aging and Physical Activity · doi:10.1123/japa.2020-0482 · Cited by 31
“The Prediction model Risk of Bias Assessment tool includes 20 signaling questions grouped within four domains (participant selection, predictors, outcome, and analysis) to evaluate the risk of bias of prediction models (Wolff et al., 2019).”
MethodPrediction Models for Future High-Need High-Cost Healthcare Use: a Systematic Review
de Ruijter, U. W., Kaplan, Z. L. R., Bramer, W. M., Eijkenaar, F., Nieboer, D., van der Heide, A., Lingsma, H. F., & Bax, W. A. (2022) · Journal of General Internal Medicine · doi:10.1007/s11606-021-07333-z · Cited by 34
“(e) Risk of bias as assessed through Prediction model Risk Of Bias Assessment Tool (PROBAST).(28) U = outcome based on utilization; C = outcome based on cost; EPV = events per variable; ROB = risk of bias; ED = emergency department.”
MethodInfluentialYang, L., Iwami, M., Chen, Y., Wu, M., & van Dam, K. H. (2022) · Progress in Planning · doi:10.1016/j.progress.2022.100657 · Cited by 26
“ABM + Epidemiological Modelling Small and Cavanagh (2020) Comp. Pred.”
InfluentialRaveendran, D., Penny‐Dimri, J. C., Segal, R., Smith, J. A., Plummer, M. P., Liu, Z., & Perry, L. A. (2022) · Journal of Cardiothoracic Surgery · doi:10.1186/s13019-022-01870-2 · Cited by 26
“PROBAST is tailored for prognostic studies and assesses risk of bias across four domains: participants, predictors, outcome, and analysis [16, 17].”
MethodAlatrany, A. S., Hussain, A. J., Mustafina, J., & Al‐Jumeily, D. (2022) · IEEE Access · doi:10.1109/access.2022.3182543 · Cited by 31
Huang, B. B., Huang, J., & Swong, K. N. (2022) · World Neurosurgery · doi:10.1016/j.wneu.2022.08.109 · Cited by 29
Zhou, Y. J., Yang, X., Ma, S., Yuan, Y., & Yan, M. (2022) · Nursing Open · doi:10.1002/nop2.1429 · Cited by 29
Karabacak, M., Ozkara, B. B., Öztürk, A., Kaya, B., Cirak, Z., Orak, E., & Ozcan, Z. (2022) · Acta Radiologica · doi:10.1177/02841851221143496 · Cited by 27
“Although efforts are being made to standardize the methodological approach, their application outside academic research has not yet been proven appropriate (39 – 41).”
BackgroundResultChen, F., Kantagowit, P., Nopsopon, T., Chuklin, A., & Pongpirul, K. (2023) · PLoS ONE · doi:10.1371/journal.pone.0278729 · Cited by 29
“We will assess the risk of bias (ROB) of studies by applying PROBAST (prediction model risk of bias assessment tool) [34].”
MethodQi, W., Wang, Y., Li, C., He, K., Wang, Y., Huang, S., Li, C., Guo, Q., & Hu, J. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.04.026 · Cited by 32
Groot, T. M., Ramsey, D. C., Groot, O. Q., Fourman, M. S., Karhade, A. V., Twining, P. K., Berner, E. A., Fenn, B. P., Collins, A. K., Raskin, K. A., et al. (2023) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000002698 · Cited by 29
“However, the missForest method for multiple imputation is considered an appropriate approach to handle missing data [15, 28].”
MethodThe State of Artificial Intelligence in Pediatric Surgery: A Systematic Review
Elahmedi, M. O., Sawhney, R., Guadagno, E., Botelho, F. M. N., & Poenaru, D. (2024) · Journal of Pediatric Surgery · doi:10.1016/j.jpedsurg.2024.01.044 · Cited by 26
Training Deep Networks from Zero to Hero: avoiding pitfalls and going beyond
Ponti, M. A., dos Santos, F. P., Ribeiro, L. S. F., & Cavallari, G. B. (2021) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi54419.2021.00011 · Cited by 32
Artificial Intelligence developments in medical education: a conceptual and practical framework
Masters, K. (2020) · MedEdPublish · doi:10.15694/mep.2020.000239.1 · Cited by 25
“Various suggestions have been made to reduce bias, including the development of tools and models such as PROBAST (Wolff et al. , 2019), and medical practitioners will also need to be cognisant of data usage and security laws, such as GDPR, the 21st Century Cures Act (USA Hse of Rep, 2016) and…”
MethodExploring Machine Learning Applications in Pediatric Asthma Management: Scoping Review
Ojha, T., Patel, A., Sivapragasam, K., Sharma, R., Vosoughi, T., Skidmore, B., Pinto, A. D., & Hosseini, B. (2024) · JMIR AI · doi:10.2196/57983 · Cited by 31
“The risk of bias in the included studies was assessed using the PROBAST tool [44].”
Methodvan Eekhout, J. C. A., Becking, E. C., Scheffer, P. G., Koutsoliakos, I., Bax, C. J., Henneman, L., Bekker, M. N., & Schuit, E. (2024) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17983 · Cited by 27
Li, W. W., Nannestad, J., Leow, T. Y. Q., & Heward, C. (2024) · Health Psychology Open · doi:10.1177/20551029241302969 · Cited by 27
“An assessment of the risk of bias for each included study was conducted employing the Prediction Model Study Risk of Bias Assessment Tool (PROBAST; Wolff et al., 2019).”
Williams, E., Huynh, D., Estai, M., Sinha, T., Summerscales, M., & Kanagasingam, Y. (2025) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2025.100197 · Cited by 27
Risk prediction models for adolescent suicide: A systematic review and meta-analysis
Li, R., Yue, Y., Gu, X., Xiong, L., Luo, M., & Li, L. (2025) · Psychiatry Research · doi:10.1016/j.psychres.2025.116405 · Cited by 30
Critelli, B., Hassan, A., Lahooti, I., Noh, L., Park, J. S., Tong, K., Lahooti, A., Matzko, N. E., Adams, J. N., Liß, L., et al. (2025) · PLoS Medicine · doi:10.1371/journal.pmed.1004432 · Cited by 29
“The PROBAST ROB tool dedicates nine signaling questions to this domain [42].”
Vermeulen, R. J., Andersson, V., Banken, J., Hannink, G., Govers, T. M., Rovers, M. M., & Rikkert, M. O. (2025) · Alzheimer s & Dementia · doi:10.1002/alz.70069 · Cited by 29
Smiley, A., Villarreal‐Zegarra, D., Reátegui-Rivera, C. M., Escobar-Agreda, S., & Finkelstein, J. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1555247 · Cited by 27
“The PROBAST tool assessed the risk of bias (RoB) in studies on prediction models for diagnosis, prognosis, and treatment.”
InfluentialNayebirad, S., Hassanzadeh, A., Vahdani, A. M., Mohamadi, A., Forghani, S., Shafiee, A., & Masoudkabir, F. (2025) · BMC Cardiovascular Disorders · doi:10.1186/s12872-025-04746-0 · Cited by 26
“Critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS) checklist [14] and prediction model risk of bias assessment tool (PROBAST) [15] were used by two independent reviewers (S.N and A.”
Martin, G. P., Riley, R. D., Ensor, J., & Grant, S. W. (2025) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezaf142 · Cited by 35
“Additionally, there is guidance to aid the critical appraisal of CPM studies [47, 48], and guidance on reporting CPM studies through the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) checklist, and associated explanation and elaboration…”
Self-citationPrediction models for self-harm and suicide: a systematic review and critical appraisal
Seyedsalehi, A., Bailey, J. P. M., Ogonah, M. G. T., Fanshawe, T. R., & Fazel, S. (2025) · BMC Medicine · doi:10.1186/s12916-025-04367-6 · Cited by 29
“A standardised data extraction and risk of bias assessment form was developed based on the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Model-ling Studies (CHARMS) [27] and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) [28, 29].”
Heestermans, T., Payne, B. A., Kayode, G. A., Amoakoh‐Coleman, M., Schuit, E., Rijken, M. J., Klipstein‐Grobusch, K., Bloemenkamp, K. W. M., Grobbee, D. E., & Browne, J. L. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2019-001759 · Cited by 28
“risk of bias in individual studies The PROBAST (PRediction model Risk Of Bias ASsessment Tool) of risk assessment was used to define the risk of bias.(17) The following themes were assessed: patient selection, predictors, outcomes and analysis.”
MethodToolkit of methodological resources to conduct systematic reviews
Figuls, M. R. I., García, L. M., Solà, I., Alonso‐Coello, P., Bonfill, X., & Zamora, J. (2020) · F1000Research · doi:10.12688/f1000research.22032.2 · Cited by 28
Multimorbidity measures from health administrative data using ICD system codes: A systematic review
Simard, M., Rahme, E., Calfat, A. C., & Sirois, C. (2021) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5368 · Cited by 30
Vannucci, M., Laracca, G. G., Mercantini, P., Perretta, S., Padoy, N., Dallemagne, B., & Mascagni, P. (2021) · Surgery · doi:10.1016/j.surg.2021.10.001 · Cited by 31
Risk prediction models for breast cancer: a systematic review
Zheng, Y., Li, J., Wu, Z. L., Li, H., Cao, M., Li, N., & He, J. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-055398 · Cited by 30
Siddiqi, T. J., Ahmed, A. B., Greene, S. J., Shahid, I., Usman, M., Oshunbade, A. A., Alkhouli, M. A., Hall, M. E., Murad, M. H., Khera, R., et al. (2022) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwac148 · Cited by 25
Koch, A. H., Jeelof, L. S., Muntinga, C. L. P., Gootzen, T. A., van de Kruis, N. M. A., Nederend, J., Boers, T. G. W., van der Sommen, F., & Piek, J. M. J. (2023) · Insights into Imaging · doi:10.1186/s13244-022-01345-x · Cited by 26
“Studies that evaluated diagnostic tests of prognostic factors were reviewed by a few questions from the QUADAS-2 tool and the QUIPS tool [27–30].”
MethodHossain, M. A., & Ahmed, F. (2023) · doi:10.63125/mqty9n77 · Cited by 26
Connell, L., Chesworth, B. M., Ackerley, S., Smith, M., & Stinear, C. M. (2021) · Physical Therapy · doi:10.1093/ptj/pzab040 · Cited by 27
Lozanovski, V. J., Probst, P., Arefidoust, A., Ramouz, A., Aminizadeh, E., Nikdad, M., Khajeh, E., Ghamarnejad, O., Shafiei, S., Ali‐Hasan‐Al‐Saegh, S., et al. (2021) · Transplant International · doi:10.1111/tri.13861 · Cited by 26
“The risk of bias, the study methodology, and the relevance of the findings to the research question (applicability) were rated “high,” “low,” or “unclear” based on a predefined questionnaire and scoring guide [16].”
BackgroundPredictive Modeling for Readmission to Intensive Care: A Systematic Review
Ruppert, M. M., Loftus, T. J., Small, C., Li, H., Ozrazgat-Baslanti, T., Balch, J., Holmes, R., Tighe, P. J., Upchurch, G. R., Efron, P. A., et al. (2023) · Critical Care Explorations · doi:10.1097/cce.0000000000000848 · Cited by 24
“Study information was extracted from each of the relevant articles using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist; bias was evaluated using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) (12, 13).”
MethodHendriks, M. P., Jager, A., Ebben, K. C. W. J., van Til, J. A., & Siesling, S. (2024) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2024.104267 · Cited by 24
Systematic Review of Prognosis Models in Predicting Tooth Loss in Periodontitis
Chow, D. Y., Tay, J. R. H., & Nascimento, G. G. (2024) · Journal of Dental Research · doi:10.1177/00220345241237448 · Cited by 26
“The PROBAST tool (Prediction model Risk Of Bias ASsessment Tool) facilitates risk of bias assessment for prediction models (Wolff et al. 2019), while the TRIPOD statement (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) establishes reporting…”
MethodInfluentialGrubic, N., Johnston, A., Randhawa, V. K., Humphries, K. H., Rosella, L. C., & Maximova, K. (2024) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2024.12.022 · Cited by 35
Salman, S. G., Phadke, R. A., Kumar, R., Momin, A. A., & Tavakkoli, A. (2026) · Spine Deformity · doi:10.1007/s43390-026-01365-3 · Cited by 29
Muttalib, F., Clavel, V., Yaeger, L., Shah, V. S., & Adhikari, N. K. J. (2020) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2020.05.016 · Cited by 29
Walsh, M. E., French, H., Wallace, E., Madden, S., King, P., Fahey, T. P., & Galvin, R. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.02.005 · Cited by 24
Prediction Models for Prognosis of Cervical Cancer: Systematic Review and Critical Appraisal
He, B., Chen, W., Liu, L., Hou, Z., Zhu, H., Cheng, H., Zhang, Y., Zhan, S., & Wang, S. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.654454 · Cited by 28
“The assessment of ROB in PROBAST involved four domains (participants, predictors, outcome, and analysis), and 20 signaling questions covered the key aspects of prediction model studies.”
InfluentialOhbe, H., Goto, T., Nakamura, K., Matsui, H., & Yasunaga, H. (2022) · Intensive Care Medicine · doi:10.1007/s00134-022-06688-z · Cited by 24
“This study also meets the recommended approaches when reporting the development of a new prediction model from the Society of Critical Care Medicine [36], TRIPOD guideline [19], and Prediction Model Study Risk Of Bias Assessment Tool statement (PROBAST) [37].”
MethodPrediction models for functional status in community dwelling older adults: a systematic review
Grootven, B. V., & van Achterberg, T. (2022) · BMC Geriatrics · doi:10.1186/s12877-022-03156-7 · Cited by 25
“Adachi [18] 2018 Japan Prospective cohort Women 75 or older walking independently 79 [77 - 82] Evaluate factor 330 Mobility limitation, 2”
BackgroundMethodLi, Y., Xie, F., Xiong, Q., Lei, H., & Feng, P. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.946038 · Cited by 25
“The PROBAST quality assessment revealed some other issues that could be avoided in future studies.”
InfluentialHueting, T. A., van Maaren, M. C., Hendriks, M. P., Koffijberg, H., & Siesling, S. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.10.016 · Cited by 25
Breast Cancer Risk Assessment Tools for Stratifying Women into Risk Groups: A Systematic Review
Velentzis, L. S., Freeman, V., Campbell, D. J., Hughes, S., Luo, Q., Steinberg, J., Egger, S. J., Mann, G. B., & Nickson, C. (2023) · Cancers · doi:10.3390/cancers15041124 · Cited by 29
“Risk of bias was assessed using the ‘Prediction model Risk Of Bias ASsessment Tool’ (PROBAST), specifically designed to assess the Risk of Bias for, and the applicability of, diagnostic and prognostic prediction model studies [24].”
BackgroundMethodPredictive models in emergency medicine and their missing data strategies: a systematic review
Arnaud, É., Elbattah, M., Ammirati, C., Dequen, G., & Ghazali, D. A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00770-6 · Cited by 24
Machine Learning in X-ray Diagnosis for Oral Health: A Review of Recent Progress
Martins, M. V., Baptista, L., Luís, H., Assunção, V. A., Araújo, M., & Realinho, V. (2023) · Computation · doi:10.3390/computation11060115 · Cited by 27
“While existing standards, such as TRI-POD [54] and PROBAST [55], might not perfectly fit ML research in medical imaging, efforts should be made to adhere to reporting guidelines.”
MethodNazar, W., Szymanowicz, S., Nazar, K., Kaufmann, D., Wabich, E., Braun-Dullaeus, R. C., & Daniłowicz‐Szymanowicz, L. (2023) · Heart Failure Reviews · doi:10.1007/s10741-023-10357-8 · Cited by 24
“A “ + ” indicates low ROB/low concern regarding applicability; “ − ” indicates high ROB/high concern regarding applicability; and “?” indicates unclear ROB/unclear concern regarding applicability [52].”
BackgroundMethodMbuya-Bienge, C., Pashayan, N., Kazemali, C. D., Lapointe, J., Simard, J., & Nabi, H. (2023) · Cancers · doi:10.3390/cancers15225380 · Cited by 26
“The goal of this systematic review was to appraise and critically assess di ff erent prediction models incorporating a PRS used to estimate the risk of developing BC for women in the general population.”
MethodInfluentialZhong, K., An, X., Kong, Y., & Zhu, C. (2024) · Clinical Neurology and Neurosurgery · doi:10.1016/j.clineuro.2024.108225 · Cited by 27
Prognostic risk models for incident hypertension: A PRISMA systematic review and meta-analysis
Schjerven, F. E., Lindseth, F., & Steinsland, I. (2024) · PLoS ONE · doi:10.1371/journal.pone.0294148 · Cited by 24
“The risk of bias within studies was assessed independently by one reviewer (F.S.) using the ‘Prediction model Risk of Bias Assessment Tool’ (PROBAST) in a two-step process [11].”
Amanollahi, M., Jameie, M., Looha, M. A., Basti, F. A., Cattarinussi, G., Moghaddam, H. S., Camillo, F. D., Akhondzadeh, S., Pigoni, A., Sambataro, F., et al. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.06.061 · Cited by 27
Thorn, S., Güting, H., Maegele, M., Gruen, R. L., & Mitra, B. (2019) · Medicina · doi:10.3390/medicina55100653 · Cited by 24
“Table 1 demonstrates the results of applying the PROBAST to the included studies [14].”
Evidence-based medicine and machine learning: a partnership with a common purpose
Scott, I. A., Cook, D., & Coiera, E. W. (2020) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111379 · Cited by 26
Thurtle, D. R., Rossi, S. H., Berry, B., Pharoah, P. D. P., & Gnanapragasam, V. J. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029149 · Cited by 24
“bias assessment To assess the validity of eligible studies, individual studies were assessed for bias using the new Prediction model Risk Of Bias ASsessment Tool (PROBAST).(19) The PROBAST tool assesses both risk of bias and applicability of both model development studies and validation studies.”
Teles, A. S., de Moura, I. R., Silva, F., Roberts, A., & Stahl, D. R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102981 · Cited by 27
Vessonen, T., Dahlberg, M., Hellstrand, H., Widlund, A., Söderberg, P., Korhonen, J., Aunio, P., & Laine, A. (2025) · Review of Education · doi:10.1002/rev3.70045 · Cited by 24
Risk prediction models for esophageal cancer: A systematic review and critical appraisal
Li, H., Sun, D., Cao, M., He, S., Zheng, Y., Yu, X., Wu, Z. L., Lei, L., Peng, J., Li, J., et al. (2021) · Cancer Medicine · doi:10.1002/cam4.4226 · Cited by 23
Pivodic, A., Smith, L. E. H., Hård, A., Löfqvist, C., Almeida, A. C., Al‐Hawasi, A., Larsson, E. K. B., Lundgren, P., Sunnqvist, B., Törnqvist, K., et al. (2022) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2021-320738 · Cited by 23
WESSELS, F., Kuntz, S., KRIEGHOFF-HENNING, E., SCHMITT, M., BRAUN, V., Worst, T. S., NEUBERGER, M., STEEG, M., Gaiser, T., FRÖHLING, S., et al. (2022) · Minerva Urology and Nephrology · doi:10.23736/s2724-6051.22.04758-9 · Cited by 25
Milne‐Ives, M., Fraser, L., Khan, A., Walker, D. J., van Velthoven, M. H., May, J., Wolfe, I., Harding, T., & Meinert, E. (2022) · JMIR Research Protocols · doi:10.2196/35738 · Cited by 25
Reproducibility of prediction models in health services research
Belbasis, L., & Panagiotou, O. A. (2022) · BMC Research Notes · doi:10.1186/s13104-022-06082-4 · Cited by 28
“In addition, PROBAST, a risk-of-bias assessment tool for prediction modelling studies, can help contextualize biases arising from the selection of participants, the ascertainment of the outcome, the handling of predictors, and the statistical methods used for prediction [36, 37].”
Methodvan Ravenhorst, C. G., Schluep, M., Endeman, H., Stolker, R.-J., & Hoeks, S. E. (2023) · Critical Care · doi:10.1186/s13054-023-04306-y · Cited by 24
“Strengths and limitations This study contains a comprehensive search and extensive analysis using current guidelines for reviewing and assessing bias of prediction model studies [7, 17].”
MethodPrediction Models for Knee Osteoarthritis: Review of Current Models and Future Directions.
Ramazanian, T., Fu, S., Sohn, S., Taunton, M. J., & Kremers, H. M. (2023) · PubMed Central · doi:10.22038/abjs.2022.58485.2897 · Cited by 30
Li, X., Chen, Z., Jiao, H., Wang, B. H., Yin, H., Chen, L., Shi, H., Yin, Y., & Qin, D. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1211733 · Cited by 29
“The PROBAST consists of four domains, namely participants, predictors, outcome, and analysis (8).”
Validity of Clinical Severity Scores for Respiratory Syncytial Virus: A Systematic Review
Sheikh, Z., Potter, E., Li, Y., Cohen, R., Santos, G. D., Bont, L., Nair, H., PROMISE Investigators, Nair, H., Campbell, H., et al. (2023) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiad436 · Cited by 23
Čihařová, M., Amarti, K., van Breda, W., Peng, X., Lorente-Català, R., Funk, B., Hoogendoorn, M., Koutsouleris, N., Fusar‐Poli, P., Karyotaki, E., et al. (2024) · Biological Psychiatry · doi:10.1016/j.biopsych.2024.06.002 · Cited by 25
“Its creation was based on a review of existing validated quality assessment instruments, namely the PROBAST Tool prediction studies (33), Cochrane Risk of Bias Tool 2.0 for randomized controlled trials (34), and ROBINS-I Tool for nonrandomized studies (35), and previous reviews of ML prediction…”
MethodInfluentialPrognostic factors and prediction models for acute aortic dissection: a systematic review
Ren, Y., Huang, S., Li, Q., Liu, C., Li, L., Tan, J., Zou, K., & Sun, X. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-042435 · Cited by 23
Ashford, M. T., Veitch, D. P., Neuhaus, J. M., Nosheny, R. L., Tosun, D., & Weiner, M. W. (2021) · Alzheimer s & Dementia · doi:10.1002/alz.12253 · Cited by 23
Implementing AI models in clinical workflows: a roadmap
Wang, F., & Beecy, A. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112727 · Cited by 26
Ismail, I. N., Subramaniam, P. K., Adam, K. B. C., & Ghazali, A. B. (2024) · Diagnostics · doi:10.3390/diagnostics14171917 · Cited by 21
“PROBAST, which stands for Prediction model Risk Of Bias Assessment Tool, is a tool to assess the risk of bias and applicability of prediction model studies [64].”
Daga, K., Agarwal, S., Moti, Z., Lee, M. B. K., Din, M., Wood, D. A., Marc Modat, A., & Booth, T. C. (2024) · Clinical Neuroradiology · doi:10.1007/s00062-024-01474-4 · Cited by 24
Albalawi, F., Khanagar, S. B., Iyer, K., Alhazmi, N., Alayyash, A., Alhazmi, A. S., Awawdeh, M. A., & Singh, O. G. (2025) · Applied Sciences · doi:10.3390/app15020893 · Cited by 23
Mohammadzadeh, I., Hajikarimloo, B., Eini, P., Niroomand, B., Mohammadzadeh, S., Habibi, M. A., Babak, Z. M. S.-E., & Aliaghaei, A. (2025) · Neuroradiology · doi:10.1007/s00234-025-03657-3 · Cited by 22
“The study used the PROBAST [ 16 ] tool to assess the risk of bias (ROB) and applicability, evaluating four domains: participants, predictors, and outcomes.”
Vereijken, A. J., Aerts, I., Jetten, J., Tassignon, B., Verschueren, J., MEEUSEN, R. R., & van Trijffel, E. (2020) · VUBIR (Vrije Universiteit Brussel) · Cited by 37
Li, L., Fang, X., Cheng, L., Wang, P., Li, S., Yu, H., Zhang, Y., Jiang, N., Zeng, T., Hou, C., et al. (2021) · Aging · doi:10.18632/aging.202605 · Cited by 22
Nies, K., Smits, L. J. M., Kassem, M., Nederkoorn, P. J., van Oostenbrugge, R. J., & Kooi, M. E. (2021) · Frontiers in Neurology · doi:10.3389/fneur.2021.718438 · Cited by 24
Feeney, T., Madiedo, A. M., Knapp, P. E., Gupta, A., McAneny, D. B., & Drake, F. T. (2021) · Journal of Surgical Research · doi:10.1016/j.jss.2021.07.041 · Cited by 22
Metadata
- Authors
- Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group†
- Affiliations
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Utrecht University (NL)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- 51-58
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907554860
- Cited by
- 2,984
- References
- 42
- Citations by year
- 2026: 921 · 2025: 561 · 2024: 416 · 2023: 343 · 2022: 303 · 2021: 250 · 2020: 140 · 2019: 45
- FWCI
- 83.39
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
- Bristol Research (University of Bristol) (acceptedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
- Radboud Repository (Radboud University) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Apollo (University of Cambridge) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Apollo
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- PROBAST, risk of bias assessment, diagnostic model, prognostic model, applicability, critical appraisal, reporting guidelines, Delphi consensus, systematic reviews, personalized prediction
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Delphi Technique, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.