PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · cited 3×
“A domain-based structure was adopted similar to that used in other risk of bias tools such as the revised Cochrane Risk of Bias tool,(29) QUADAS-2,(36) ROBINSI(30) and ROBIS.”
BackgroundMethodSelf-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · cited 3×
“A domain-based structure was adopted similar to that used in other risk of bias tools such as the revised Cochrane Risk of Bias tool,(29) QUADAS-2,(36) ROBINSI(30) and ROBIS.”
InfluentialSelf-citationCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147 · cited 11×
“A multivariable prediction model is defined as any combination or equation of two or more predictors for estimating the probability or risk for an individual.(7, 8, 33-35)”
InfluentialSelf-citation2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.11.005 · Cited by 4,945
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Circulation · doi:10.1161/01.cir.0000437741.48606.98 · Cited by 3,697
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
Self-citationAssessing Bias in Studies of Prognostic Factors
Hayden, J. A., van der Windt, D. A., Cartwright, J. L., Côté, P., & Bombardier, C. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-4-201302190-00009 · Cited by 3,287 · cited 1×
“(25, 26) The QUIPS tool has been developed to assess the risk of bias in predictor finding (prognostic factor) studies.(28) The methodological quality of studies investigating the impact of a prediction model using a comparative randomised design can be assessed using the revised Cochrane risk of bias tool (ROB 2.”
MethodCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
Self-citationSystematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Cited by
Items 901–1,000 of 1,010
Liu, Z., Perry, L. A., Penny‐Dimri, J. C., Handscombe, M., Overmars, I., Plummer, M. P., Segal, R., & Smith, J. A. (2022) · Transplant International · doi:10.3389/ti.2022.10362 · Cited by 13
Gehringer, C. K., Martin, G. P., Hyrich, K. L., Verstappen, S. M. M., & Sergeant, J. C. (2022) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2022.152076 · Cited by 14
How to Develop and Validate Prediction Models for Orthopedic Outcomes
Zaniletti, I., Larson, D. R., Lewallen, D. G., Berry, D. J., & Kremers, H. M. (2022) · The Journal of Arthroplasty · doi:10.1016/j.arth.2022.12.032 · Cited by 13
Visscher, J., Hiwase, M. D., Bonevski, B., & O’Callaghan, M. E. (2023) · Prostate Cancer and Prostatic Diseases · doi:10.1038/s41391-023-00701-2 · Cited by 13
“1).”
Cronin, M., Seher, M., Arsang-Jang, S., Lowery, A., Kerin, M. J., Wijns, W. C., & Soliman, O. I. I. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12196295 · Cited by 15
“The risk of bias and reporting transparency of the included studies was assessed using the PROBAST tool [58].”
MethodGuidelines and Standard Frameworks for AI in Medicine: Protocol for a Systematic Literature Review
Shiferaw, K. B., Roloff, M., Waltemath, D., & Zeleke, A. A. (2023) · JMIR Research Protocols · doi:10.2196/47105 · Cited by 13
Spencer, K. L., Absolom, K. L., Allsop, M. J., Relton, S. D., Pearce, J., Liao, K., Naseer, S., Salako, O., Howdon, D., Hewison, J., et al. (2023) · JCO Clinical Cancer Informatics · doi:10.1200/cci.23.00070 · Cited by 14
Ren, Z., Chen, B., Hong, C., Yuan, J., Deng, J., Chen, Y., Ye, J., & Li, Y. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1289050 · Cited by 13
Li, Y., Zhang, R., Dong, L., Shi, X., Zhou, W., Wu, H., Li, H., Yu, C., & Wei, W. (2024) · Eye · doi:10.1038/s41433-023-02914-0 · Cited by 14
“Based on PROBAST (prediction model risk of bias assessment tool) [23, 24], we used a modified version that recorded items except for applicability and predictor variables, the same as the items used in a methodology review published previously [21].”
Factors determining generalization in deep learning models for scoring COVID-CT images
Horry, M. J., Chakraborty, S., Pradhan, B., Fallahpoor, M., Chegeni, H., & Paul, M. (2021) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2021456 · Cited by 14
Calvillo-Batllés, P., Cerdá-Alberich, L., Fonfría-Esparcia, C., Carreres-Ortega, A., Muñoz-Núñez, C. F., Trilles-Olaso, L., & Martí‐Bonmatí, L. (2022) · Radiología (English Edition) · doi:10.1016/j.rxeng.2021.09.004 · Cited by 15
Best Practices in Supervised Machine Learning: A Tutorial for PsychologistsPreprint
Pargent, F., Schoedel, R., & Stachl, C. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/89snd · Cited by 13
The use and misuse of risk prediction tools for clinical decision-making
van Maaren, M. C., Hueting, T. A., Völkel, V., van Hezewijk, M., Strobbe, L. J. A., & Siesling, S. (2023) · The Breast · doi:10.1016/j.breast.2023.01.006 · Cited by 14
“In addition, for users of the tool the methodology should be clearly described according to the TRIPOD guidelines [13], and risk of bias should have been minimised according to the PROBAST tool [14].”
MethodWang, M., Wang, M., Zhu, Q., Yao, X., Heizhati, M., Cai, X., Ma, Y., Wang, R., Hong, J., Yao, L., et al. (2022) · Risk Management and Healthcare Policy · doi:10.2147/rmhp.s374339 · Cited by 14
“This article was prepared in accordance with the TRIPOD reporting checklist and the methods were assessed with PROBAST.31,32”
MethodOtotoxicity prognostic models in adult and pediatric cancer patients: a rapid review
Debacker, J. R., McMillan, G. P., Martchenke, N., Lacey, C., Stuehm, H. R., Hungerford, M. E., & Konrad‐Martin, D. (2023) · Journal of Cancer Survivorship · doi:10.1007/s11764-022-01315-8 · Cited by 12
“PROBAST signaling question 4.5 (column 22 of online supplemental table) asks, “Was selection of predictors based on univariable analysis avoided?””
MethodInfluentialArtificial Intelligence and Predictive Modeling in Spinal Oncology: A Narrative Review
Kuijten, R. H., Zijlstra, H., Groot, O. Q., & Schwab, J. H. (2023) · The International Journal of Spine Surgery · doi:10.14444/8500 · Cited by 14
Population Pharmacokinetics of Isavuconazole in Adult: A Systematic Review
Chen, N., Wang, X., Li, Y., Yang, P., Huang, M., & Lu, X. (2023) · Infection and Drug Resistance · doi:10.2147/idr.s434622 · Cited by 12
Baek, S., Jeong, Y. J., Kim, Y., Kim, J. Y., Kim, J. H., Kim, E. Y., Lim, J., Kim, J., Kim, Z., Kim, K., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/52134 · Cited by 13
“…Learning Predictive Models in Biomedical Research [14], Transparent Reporting of a multivariable prediction model for Individual Prognosis (TRIPOD; File S1 in Multimedia Appendix 1), and Diagnosis and Prediction Model Risk of Bias Assessment Tool (PROBAST; File S2 in Multimedia Appendix 1) [15].”
Challenges and Potential of Artificial Intelligence in Neuroradiology
Winder, A. J., Stanley, E. A. M., Fiehler, J., & Forkert, N. D. (2024) · Clinical Neuroradiology · doi:10.1007/s00062-024-01382-7 · Cited by 13
“…intelligence in medical imaging (CLAIM) [21], minimum information about clinical artificial intelligence modeling (MI-CLAIM) [22], t reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) [23], and prediction model risk of bias assessment tool (PROBAST) [24].”
Artificial Intelligence in Imaging in the First Trimester of Pregnancy: A Systematic Review
Umans, E., Dewilde, K., Williams, H., Deprest, J. A., & den Bosch, T. V. (2024) · Fetal Diagnosis and Therapy · doi:10.1159/000538243 · Cited by 12
Risk prediction algorithms for prolonged postoperative ileus: A systematic review
Liu, G. X. H., Milne, T. G. E., Xu, W., Varghese, C., Keane, C., O’Grady, G., Bissett, I. P., & Wells, C. I. (2024) · Colorectal Disease · doi:10.1111/codi.17010 · Cited by 14
“All included studies were assessed using the Prediction model Risk Of Bias ASsessment (PROBAST) Tool [ 23,24], a quality appraisal tool that identifies risk of bias (RoB) and applicability concerns in diagnostic and prognostic prediction model studies.”
MethodSun, Y., Zhou, Y., Dai, Y., Pan, Y., Xiao, Y., & Yu, Y. (2024) · Journal of Tissue Viability · doi:10.1016/j.jtv.2024.07.002 · Cited by 15
A systematic review on artificial intelligence evaluating PSMA PET scan for intraprostatic cancer
Liu, J., Cundy, T. P., Woon, D. T. S., Desai, N. B., Palaniswami, M. S., & Lawrentschuk, N. (2024) · British Journal of Urology · doi:10.1111/bju.16412 · Cited by 13
“The Prediction model Risk Of Bias Assessment Tool (PROBAST) was utilised to assess the ROB and the applicability of diagnostic model studies [19].”
Diagnostic Performance of Machine Learning-based Models in Neonatal Sepsis: A Systematic Review
Kainth, D., Prakash, S., & Sankar, M. J. (2024) · The Pediatric Infectious Disease Journal · doi:10.1097/inf.0000000000004409 · Cited by 12
Chen, S., Yang, X., Gu, H., Wang, Y., Xu, Z., Jiang, Y., & Wang, Y. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02331-1 · Cited by 12
“The suitability of ML research was evaluated based on TRIPOD and PROBAST guidelines [30, 31].”
MethodAllotey, J., Archer, L., Coomar, D., Snell, K. I. E., Smuk, M., Oakey, L. A., Haqnawaz, S., Betrán, A. P., Chappell, L. C., Ganzevoort, W., et al. (2024) · Health Technology Assessment · doi:10.3310/dabw4814 · Cited by 12
Self-citationGoffart, S., Delingette, H., Chierici, A. P., Guzzi, L., Nasr, B., Lareyre, F., & Raffort, J. (2025) · Angiology · doi:10.1177/00033197241310572 · Cited by 15
“…in this scoping review, only a few reported the use of guidelines and included: the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE), 34 the Prediction model Risk Of Bias ASsessment Tool (PRO-BAST) 30,49 and the MINimum Information for Medical AI Reporting (MINIMAR).”
Chamarthi, B., Polu, O. R., Anumula, S. K., Ushmani, A., Kasralikar, P., & Syed, A. (2025) · Cureus · doi:10.7759/cureus.82796 · Cited by 14
“To evaluate the methodological rigor and internal validity of the included studies, the Prediction model Risk Of Bias ASsessment Tool (PROBAST) [12] was utilized.”
Wang, Y., Zheng, R., Wu, Y., Liu, T., Hao, L., Liu, J., Shi, L., & Guo, Q. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105094 · Cited by 16
External Validation of AI Models for Skin Diseases: A Systematic Review
Fernandes, T. R. S., Teles, A. S., Fernandes, J. R. N., Lima, L. D. B., Sousa, D. L., Soares, R. D. C., & Teixeira, S. S. (2025) · IEEE Access · doi:10.1109/access.2025.3584904 · Cited by 13
Pantelis, A. G., Epiphaniou, P., & Lapatsanis, D. P. (2025) · Artificial Intelligence Surgery · doi:10.20517/ais.2024.104 · Cited by 13
Ndjonko, L. C. M., Chakraborty, A., Petri, F., Alavi, S. M. A., Matsuo, T., Borgonovo, F., Comba, I. Y., Murad, M. H., Nassr, A. N., Zein, S. E., et al. (2025) · The Spine Journal · doi:10.1016/j.spinee.2025.07.032 · Cited by 14
O’Toole, C. C., Boakye, N. F., Hannigan, A., & Jalali, A. (2025) · Prostate Cancer and Prostatic Diseases · doi:10.1038/s41391-025-01014-2 · Cited by 15
“Risk of bias was assessed using the Risk Of Bias ASsessment Tool (PROBAST) for prediction models [29] by two independent reviewers, with con fl ict resolution as needed.”
Ardila, C. M., Vivares‐Builes, A. M., & Yadalam, P. K. (2025) · Medical Sciences · doi:10.3390/medsci13030159 · Cited by 14
Kim, Y., Oh, S., & Kim, G. (2025) · Signals · doi:10.3390/signals6040051 · Cited by 17
“Where patient-level risk models or digital-twin surrogates are evaluated, TRIPOD and PROBAST help ensure transparent reporting and risk-of-bias assessment [227,228].”
Ivanovic, V., Sujan, M. A. J., Mengshoel, O. J., & Moholdt, T. T. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106158 · Cited by 11
Garcia-Atutxa, I., Villanueva‐Flores, F., Barrio, E. D., Sanchez-Villamil, J. I., Martínez-Más, J., & Bueno-Crespo, A. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1649746 · Cited by 15
Jabed, M. I. K., Imran, M., Gomes, C. A., & Ponduru, P. S. (2025) · World Journal of Advanced Research and Reviews · doi:10.30574/wjarr.2025.27.1.2649 · Cited by 14
Langerak, A. J., McCambridge, A. B., Stubbs, P. W., Fabricius, J., Rogers, K. D., de Oliveira, C. Q., Nielsen, J. F., & Verhagen, A. P. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.03.022 · Cited by 13
“Two assessors (AL, AV) independently assessed risk of bias (RoB) using the Prediction model Risk Of Bias Assessment Tool (PROBAST) [15] .”
MethodSoeteman, M., Lekkerkerker, C. W. M., Kappen, T. H., Tissing, W. J. E., Nieuwenhuis, E. E. S., & Asperen, R. M. W. (2022) · Pediatric Blood & Cancer · doi:10.1002/pbc.29636 · Cited by 13
Jutidamrongphan, W., Kritpracha, B., Sörelius, K., Chichareon, P., Chongsuvivatwong, V., Sungsiri, J., Rookkapan, S., Premprabha, D., Juntarapatin, P., Tantarattanapong, W., et al. (2022) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2022.11.003 · Cited by 12
Yuan, E., Chen, Y., & Song, B. (2023) · European Radiology · doi:10.1007/s00330-023-09414-5 · Cited by 13
“We then used PROBAST, also designed for multivariable prediction model, to assess the risk of bias in these models from four domains: participants, predictors, outcome, and analysis [43].”
MethodZhang, S., Zhang, K., Chen, W. Y., & Wu, C. (2023) · Health Science Reports · doi:10.1002/hsr2.1309 · Cited by 13
Langeslag, J. F., Berendse, K., Daams, J. G., Onland, W., Leeflang, M., van Kaam, A. H., & de Haan, T. R. (2023) · Neonatology · doi:10.1159/000530411 · Cited by 13
“Future studies should follow the state-of-the-art guidelines for prediction model development to improve the quality of the derivation and reporting of new prediction models [13].”
BackgroundMethodInfluentialYan, A., Samarawickrema, I., Naunton, M., Peterson, G. M., Yip, D., Newman, P., & Mortazavi, R. (2024) · Thrombosis Research · doi:10.1016/j.thromres.2024.01.003 · Cited by 12
Salehi, M. A., Mohammadi, S., Harandi, H., Zakavi, S. S., Jahanshahi, A., Farahani, M. S., & Wu, J. S. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-023-00945-3 · Cited by 13
“To evaluate bias and applicability, we utilized the Prediction Model Study Risk of Bias Assessment Tool (PROBAST) checklist, which is designed to assess papers based on four key areas: participants, predictors, outcomes, and analysis (Table E7) [12].”
Use of ROC curve analysis for prediction gives fallacious results: Use predictivity-based indices
Indrayan, A., Malhotra, R., & Pawar, M. (2024) · Journal of Postgraduate Medicine · doi:10.4103/jpgm.jpgm_753_23 · Cited by 12
“[23] ).”
Gates, M., Pillay, J., Thériault, G., Limburg, H., Grad, R. M., Klarenbach, S. W., Korownyk, C. S., Reynolds, D. L., Riva, J. J., Thombs, B. D., et al. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1094-5 · Cited by 13
“The 2018 USPSTF used the Cochrane Risk of Bias Tool [129] to assess randomized controlled trials (KQ1 and KQ3a) and the Prediction model Risk Of Bias Assessment Tool [130, 131] to assess prognostic accuracy studies (KQ2).”
MethodBurlacu, A., Iftene, A., Popa, I. V., Crișan-Dabija, R. A., Brinza, C., & Covic, A. C. (2021) · Medicina · doi:10.3390/medicina57060538 · Cited by 12
“We assessed the risk of bias for non-randomized studies by applying PROBAST (prediction model risk of bias assessment tool) [20].”
MethodSimon-Pimmel, J., Foucher, Y., Léger, M., Feuillet, F., Bodet-Contentin, L., Cinotti, R., Frasca, D., & Dantan, É. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047279 · Cited by 13
“Beyond all methodological issues, patient-centred clinical finality should guide prognostic tools to be convincing.”
Modern Learning from Big Data in Critical Care: Primum Non Nocere
Gravesteijn, B. Y., Steyerberg, E. W., & Lingsma, H. F. (2022) · Neurocritical Care · doi:10.1007/s12028-022-01510-6 · Cited by 14
“An important bias in these studies is selection bias, for example, arising because of inappropriate handling of missing data [87].”
BackgroundPrediction of mortality in premature neonates. An updated systematic review
del Río, R., Thió, M., Bosio, M., Figueras, J., & Iriondo, M. (2020) · Anales de Pediatría (English Edition) · doi:10.1016/j.anpede.2019.11.005 · Cited by 12
Yan, A., Samarawickrema, I., Naunton, M., Peterson, G. M., Yip, D., Rosa, S. D., & Mortazavi, R. (2021) · Healthcare · doi:10.3390/healthcare9060778 · Cited by 13
“We used the Prediction model Risk Of Bias ASessment Tool (PROBAST) [64] to identify potential biases in the development or validation of current VTE risk models.”
MethodApplications of artificial intelligence in vasculitides: a systematic review
Omar, M., Agbareia, R., Naffaa, M. E., Watad, A., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2025) · ACR Open Rheumatology · doi:10.1002/acr2.70016 · Cited by 11
Leveraging GANs for data scarcity of COVID-19: Beyond the hype
Ali, H., Grönlund, C., & Shah, Z. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00073 · Cited by 18
“The research community (including the authors, the reviewers, and the readers) should consider evaluating their work against the Radiological Society of North America Checklist (RSNA CLAIM) [52], assessment tools like PROBAST [53], or similar metrics, that provide a comprehensive checklist for AI models from the persp…”
Jiang, H., Chen, R., Li, Y., Hao, C., Song, G., Hua, Z., Li, J., Wang, Y., & Wei, W. (2023) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000002629 · Cited by 12
Russian registry of Surgical OutcomeS — RuSOS: study protocol
Zabolotskikh, I. B., Belkin, A. A., Grigoryev, E. V., Grigoryev, S. V., Gritsan, A. I., Вадимович, Д. П., Ershov, V. I., Кirov, М. Y., Kuzovlev, A. N., Kulikov, A. V., et al. (2024) · Annals of critical care · doi:10.21320/1818-474x-2024-1-158-167 · Cited by 11
Sun, Y., Liu, Y., Zhu, Y., Luo, R., Luo, Y., Wang, S., & Feng, Z. (2024) · Current Medical Research and Opinion · doi:10.1080/03007995.2024.2307346 · Cited by 14
“In this investigation, the Prediction Model Risk of Bias Assessment Tool (PROBAST) was employed to systematically evaluate the potential bias and applicability of multifactorial prediction models 23,24 .”
MethodApplication of Machine Learning in Multimorbidity Research: Protocol for a Scoping Review
Anthonimuthu, D. J., Hejlesen, O. K., Zwisler, A., & Udsen, F. W. (2024) · JMIR Research Protocols · doi:10.2196/53761 · Cited by 14
“On the other hand, using tools such as the Prediction Model Risk of Bias Assessment, which is designed for systematic reviews, could provide insights into the quality of prediction models [51].”
MethodDiao, H., Lu, G., Wang, Z., Zhang, Y., Liu, X., Ma, Q., Yu, H., & Li, Y. (2024) · Clinical Neurology and Neurosurgery · doi:10.1016/j.clineuro.2024.108430 · Cited by 11
Prognostic prediction models for adverse birth outcomes: A systematic review
Muche, A. A., Baruda, L. L., Pons‐Duran, C., Fite, R. O., Gelaye, K. A., Worku, A., Taddesse, L., Bekele, D., Tolera, G., Chan, G. J., et al. (2024) · Journal of Global Health · doi:10.7189/jogh.14.04214 · Cited by 14
“Each domain was rated as having a high, low, or unclear risk of bias [22].”
Knowledge of the genetics of human pain gained over the last decade from next-generation sequencing
Kringel, D., & Lötsch, J. (2025) · Pharmacological Research · doi:10.1016/j.phrs.2025.107667 · Cited by 12
Advances in risk prediction models for cancer-related cognitive impairment
Duan, R., Wen, Z., Zhang, T., Liu, J., Feng, T., & Ren, T. (2025) · Clinical and Experimental Medicine · doi:10.1007/s10238-025-01590-6 · Cited by 13
“The PROBAST results [32] indicate that all nine included studies had a high risk of bias, with the majority of biases concentrated in the analysis domain (Table 2).”
de Reus, D. C., Kuijten, R. H., Saha, P., Lastoria, D. A. A., Warr-Esser, A., Taylor, C. F. C., Groot, O. Q., Lui, D. F., Verlaan, J., & Tobert, D. G. (2025) · The Spine Journal · doi:10.1016/j.spinee.2025.03.018 · Cited by 12
Tian, C., Chen, H., Shao, W., Zhang, R., Yao, X., & Shu, J. (2025) · European journal of medical research · doi:10.1186/s40001-025-02545-z · Cited by 11
“The risk of bias in the primary study was assessed by Prediction model Risk of Bias Assessment Tool (PROBAST) [22].”
Predictive accuracy of ophthalmic artery Doppler for pre-eclampsia: a systematic review
Arkorful, J., Browne, J. L., Adu‐Bonsaffoh, K., Ali, S., Srofenyoh, E., & Bloemenkamp, K. W. M. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-094348 · Cited by 13
Bull, C. F., Kisely, S. R., Betts, K. S., & Hu, Y. (2025) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2025.107630 · Cited by 13
Sample Selection Bias in Machine Learning for Healthcare
Chauhan, V. K., Clifton, L., Salaün, A., Lu, H. Y., Branson, K., Schwab, P., Nigam, G. B., & Clifton, D. A. (2025) · ACM Transactions on Computing for Healthcare · doi:10.1145/3761822 · Cited by 13
“Wolff et al. [69] developed a critical appraisal tool called the Prediction Algorithm Risk of Bias Assessment Tool (PROBAST), to assess the risk of bias and applicability of diagnostic and prognostic prediction algorithms.”
Predictive models for chemotherapy-induced oral mucositis: a systematic review
Tao, Y., Zeng, X. G., & Mao, H. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1608505 · Cited by 11
Zhao, Z., Yang, L., Quan, Q., Wang, H., Zhang, W., & Zhang, X. (2025) · International Urology and Nephrology · doi:10.1007/s11255-025-04795-6 · Cited by 12
“PROBAST evaluates four domains: participant selection, predictor variables, outcome measures, and analytical approach [20].”
Abuzuhri, M.-A., Najjar, S., Awad, M., Cruz-Correia, R. J., Falna, H., Aktaş, E., Rob, B. M. A. A., Oliveira, M., Mughnnamin, I., Esteves, S. N., et al. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106170 · Cited by 11
Mohammadzadeh, I., Hajikarimloo, B., Mohammadzadeh, S., Niaei, A. M., Sanjari, P., Habibi, M. A., Minaee, P., Dmytriw, A. A., Günkan, A., & Jabbour, P. M. (2025) · Neurosurgical Review · doi:10.1007/s10143-025-03901-7 · Cited by 11
“The PROBAST tool was used to evaluate the risk of bias (ROB) and the applicability of the included studies [ 19].”
Añez, D., Conti, G., Uriarte, J. J., Olmedo, J. J. S., Martínez‐Murillo, R., & Casanova-Carvajal, Ó. (2025) · Medicina · doi:10.3390/medicina61122237 · Cited by 12
“Although it provides an initial assessment of methodological quality, future studies could benefit from adopting structured instruments such as PROBAST [51] to perform a more systematic and rigorous evaluation of bias and applicability.”
Symptom scores in the diagnosis of pediatric cow’s milk protein allergy: A systematic review
Thompson, G., Zhelev, Z., Peters, J., Khalid, S., Briscoe, S., Shaw, L., Nunns, M., Ludman, S., & Hyde, C. (2021) · Pediatric Allergy and Immunology · doi:10.1111/pai.13537 · Cited by 12
Gao, Y., Chen, Y., Hu, M., Gan, T., Sun, X., Zhang, Z., He, W., & Wu, X. (2022) · Journal of Applied Gerontology · doi:10.1177/07334648221097084 · Cited by 11
“For predictor selection process, the PROBAST stated that the better way to include or omit candidate predictors is using non-statistical methods like existing knowledge (Wolff et al., 2019).”
BackgroundMethodInfluentialSeeking Standardized Definitions for HLA-incompatible Kidney Transplants: A Systematic Review
Jatana, S., Zhao, H., Bow, L. M., Cozzi, E., Batal, I., Horak, T. E., Amar‐Zifkin, A., Schinstock, C. A., Askar, M. Z., Dadhania, D. M., et al. (2022) · Transplantation · doi:10.1097/tp.0000000000004262 · Cited by 11
Improving risk prediction for death, stroke and bleeding in Asian patients with atrial fibrillation
Krittayaphong, R., Kanjanarutjawiwat, W., Wisaratapong, T., & Lip, G. Y. H. (2022) · European Journal of Clinical Investigation · doi:10.1111/eci.13886 · Cited by 12
Taber, P. A., Armin, J. S., Orozco, G., Fiol, G. D., Erdrich, J., Kawamoto, K., & Israni, S. T. (2023) · Current Oncology Reports · doi:10.1007/s11912-023-01376-7 · Cited by 12
“Formal quality assessment tools like QUADAS-2 [74] and PROBAST [75] are oriented toward encouraging transparency about overall model performance.”
Methodde Paiva, B. B. M., Delfino-Pereira, P., de Andrade, C. M. V., Gomes, V. M. R., Souza-Silva, M. V. R., Martins, K. P. M. P., Sales, T. L. S., de Carvalho, R. L. R., Pires, M. C., Ramos, L. E. F., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-28579-z · Cited by 15
Romijn, M., van Kaam, A. H., Fenn, D. W., Bos, L. D. J., van den Akker, C. H. P., Finken, M. J. J., Rotteveel, J., Cerullo, J., Brinkman, P., & Onland, W. (2023) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2023.02.014 · Cited by 11
Lee, T. Y., Sadatsafavi, M., Yadav, C. P., Price, D. B., Beasley, R. W., Janson, C., Koh, M. S., Roy, R., & Chen, W. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-070459 · Cited by 11
Aly, F., Hansen, C. R., Mouiee, D. A., Sundaresan, P., Haidar, A., Vinod, S. K., & Holloway, L. C. (2023) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2023.109629 · Cited by 11
Physiological and Biomechanical Monitoring in American Football Players: A Scoping Review
Nocera, A., Sbrollini, A., Romagnoli, S., Morettini, M., Gambi, E., & Burattini, L. (2023) · Sensors · doi:10.3390/s23073538 · Cited by 12
“The studies were divided into one of the following categories: sports science [40], case series [41], case-control [42], case report [43], longitudinal prospective cohort [42], before– after cohort with no control [44], descriptive laboratory design [45], quasi-experimental [46], randomized control trials [46], predic…”
BackgroundLee, T., Chen, Y., Groot, O. Q., Yen, H., Bindels, B. J. J., Pierik, R., Hsieh, H., Karhade, A. V., Tseng, T., Lai, Y., et al. (2023) · Cancer Medicine · doi:10.1002/cam4.6097 · Cited by 11
To predict the risk of chronic kidney disease (CKD) using Generalized Additive2 Models (GA2M)
Lapi, F., Nuti, L., Marconi, E., Medea, G., Cricelli, I., Papi, M., Gorini, M., Fiorani, M., Piccinocchi, G., & Cricelli, C. A. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad097 · Cited by 11
Association of adverse outcomes with three-tiered risk assessment tool for obstetrical hemorrhage
Ghose, I., Wiley, R. L., Ciomperlik, H. N., Chen, H.-Y., Sibai, B. M., Chauhan, S. P., & Figueroa, H. M. (2023) · American Journal of Obstetrics & Gynecology MFM · doi:10.1016/j.ajogmf.2023.101106 · Cited by 11
Herrera, D. J., van de Veerdonk, W., Seibert, D. M., Boke, M. M., Gutiérrez‐Ortiz, C., Yimer, N. B., Feyen, K., Ferrari, A., & Hal, G. F. V. (2023) · Gastrointestinal Disorders · doi:10.3390/gidisord5040045 · Cited by 14
“The RoB and applicability of risk prediction models from eligible studies was assessed using the PROBAST tool, following recommendations from the Cochrane Prognosis Meth-ods Group [25,26].”
Contesting the conventional wisdom of periodontal risk assessment
Raittio, E. J., López, R., & Bælum, V. (2024) · Community Dentistry And Oral Epidemiology · doi:10.1111/cdoe.12942 · Cited by 11
Hurkmans, C., Bibault, J., Clementel, E., Dhont, J., van Elmpt, W., Kantidakis, G., & Andratschke, N. H. (2024) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2024.110196 · Cited by 13
Clinical prediction models for treatment outcomes in newly diagnosed epilepsy: A systematic review
Ratcliffe, C., Pradeep, V., Marson, A. G., Keller, S. S., & Bonnett, L. J. (2024) · Epilepsia · doi:10.1111/epi.17994 · Cited by 12
“45 Similarly, three of the four domains contain an applicability concerns judgment, whereby the rater evaluates to what extent the study content matches the research question.”
Artificial Intelligence for Sport Injury Prediction
Owen, R., Owen, J. A., & Evans, S. L. (2024) · Springer eBooks · doi:10.1007/978-3-031-67256-9_5 · Cited by 14
Chao, P.-J., Chang, C.-H., Wu, J.-J., Liu, Y.-H., Shiau, J., Shih, H.-H., Lin, G.-Z., Lee, S.-H., & Lee, T.-F. (2024) · Cancer Control: Journal of the Moffitt Cancer Cente · doi:10.1177/10732748241286749 · Cited by 11
“22 The assessment covers four main domains — participants, predictors, outcomes, and analysis — which together comprise 20 signaling questions to facilitate a structured judgment of risk of bias (ROB).”
Cox, E. G. M., Meijs, D. A. M., Wynants, L., Sels, J.-W. E. M., Koeze, J., Keus, F., Dongen, B. B.-V., van der Horst, I. C. C., & van der Horst, I. C. C. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111605 · Cited by 12
Boyles, T. H., Stadelman, A., Ellis, J. P., Cresswell, F. V., Lutje, V., Wasserman, S. A., Tiffin, N., & Wilkinson, R. J. (2019) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15056.3 · Cited by 12
Xu, W., Huang, J., Yu, Q., Yu, H., Pu, Y., & Shi, Q. (2021) · Neurological Sciences · doi:10.1007/s10072-021-05219-w · Cited by 12
Risk prediction models for symptomatic patients with bladder and kidney cancer: a systematic review
Harrison, H., Usher‐Smith, J. A., Li, L., Roberts, L. E., Lin, Z., Thompson, R. E., Rossi, S. H., Stewart, G. D., Walter, F. M., Griffin, S. J., et al. (2021) · British Journal of General Practice · doi:10.3399/bjgp.2021.0319 · Cited by 11
“The PROBAST tool was used to assess risk of bias (RoB) over four domains of interest (population, risk factors, outcomes and analysis) [15, 16].”
Ding, F., Hu, K., Liu, X., Liu, C., Yang, J., Shi, X., Liu, B., Wu, M., Wang, Z., Feng, L., et al. (2022) · Regenerative Biomaterials · doi:10.1093/rb/rbac076 · Cited by 13
Rademaker, M. M., Meijers, S. M., Smit, A. L., & Stegeman, I. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12020695 · Cited by 13
“The PROBAST tool consists of 20 signalling questions divided over four domains: participants, predictors, outcome and analysis.”
MethodInfluentialKunonga, P., Kenny, R. P. W., Astin, M., Bryant, A., Kontogiannis, V., Coughlan, D., Richmond, C., Eastaugh, C., Beyer, F. R., Pearson, F., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073306 · Cited by 11
“44 This review followed procedures documented by the Cochrane Collaboration for conducting systematic reviews, the CHARMS 20 guidelines for extracting data, the PRISMA 16 guidelines for reporting and PROBAST tool 21 for assessing risk of bias, so was therefore robust.”
MethodInfluentialHarrison, H., Pennells, L. A. M., Wood, A., Rossi, S. H., Stewart, G. D., Griffin, S. J., & Usher‐Smith, J. A. (2021) · British Journal of Urology · doi:10.1111/bju.15598 · Cited by 12
Metadata
- Authors
- Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group†
- Affiliations
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Utrecht University (NL)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- 51-58
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907554860
- Cited by
- 2,984
- References
- 42
- Citations by year
- 2026: 921 · 2025: 561 · 2024: 416 · 2023: 343 · 2022: 303 · 2021: 250 · 2020: 140 · 2019: 45
- FWCI
- 83.39
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
- Bristol Research (University of Bristol) (acceptedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
- Radboud Repository (Radboud University) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Apollo (University of Cambridge) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Apollo
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- PROBAST, risk of bias assessment, diagnostic model, prognostic model, applicability, critical appraisal, reporting guidelines, Delphi consensus, systematic reviews, personalized prediction
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Delphi Technique, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.