PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · cited 3×
“A domain-based structure was adopted similar to that used in other risk of bias tools such as the revised Cochrane Risk of Bias tool,(29) QUADAS-2,(36) ROBINSI(30) and ROBIS.”
BackgroundMethodSelf-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · cited 3×
“A domain-based structure was adopted similar to that used in other risk of bias tools such as the revised Cochrane Risk of Bias tool,(29) QUADAS-2,(36) ROBINSI(30) and ROBIS.”
InfluentialSelf-citationCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147 · cited 11×
“A multivariable prediction model is defined as any combination or equation of two or more predictors for estimating the probability or risk for an individual.(7, 8, 33-35)”
InfluentialSelf-citation2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.11.005 · Cited by 4,945
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Circulation · doi:10.1161/01.cir.0000437741.48606.98 · Cited by 3,697
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
Self-citationAssessing Bias in Studies of Prognostic Factors
Hayden, J. A., van der Windt, D. A., Cartwright, J. L., Côté, P., & Bombardier, C. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-4-201302190-00009 · Cited by 3,287 · cited 1×
“(25, 26) The QUIPS tool has been developed to assess the risk of bias in predictor finding (prognostic factor) studies.(28) The methodological quality of studies investigating the impact of a prediction model using a comparative randomised design can be assessed using the revised Cochrane risk of bias tool (ROB 2.”
MethodCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
Self-citationSystematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Cited by
Items 701–800 of 1,010
Tourab, H., Lopez-Perez, L., Arroyo, P., Georga, E. I., Rujas, M., Ponziani, F. R., Torrego-Ellacuría, M., Barbancho, B. M., Dei, N. N., Ciuti, G., et al. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3593198 · Cited by 21
“The PROBAST (Prediction model Risk Of Bias ASsessment Tool) is another tool used for assessing the risk of bias (ROB) and applicability concerns in diagnostic and prognostic prediction model studies [97].”
Jahrami, H., Husain, W., Trabelsi, K., Penzel, T., Hirshkowitz, M., Razjouyan, J., BaHammam, A. S., & Sharafkhaneh, A. (2025) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2025.102212 · Cited by 22
The accuracy of statistical shape models in predicting bone shape: A systematic review
Patil, A., Kulkarni, K., Xie, S., Bull, A. M. J., & Jones, G. G. (2023) · International Journal of Medical Robotics and Computer Assisted Surgery · doi:10.1002/rcs.2503 · Cited by 18
“Risk of Bias of included studies was assessed by two independent reviewers (AP and KK), utilising the Prediction model Risk of Bias Assessment Tool (PROBAST) tool.(14) PROBAST contains a total of 20 signalling questions, divided into four key domains relevant to prediction models: Participants, Predictors, Outcomes, A…”
MethodRech, M. M., Filho, L. D. M., White, A. J., Perez‐Vega, C., Samson, S. L., Chaichana, K. L., Olomu, O. U., Quiñones‐Hinojosa, A., & Almeida, J. P. (2023) · Brain Sciences · doi:10.3390/brainsci13030495 · Cited by 22
“This scope of literature could also benefit from a formal assessment of the risk of bias of published studies, for example, with the use of PROBAST (Prediction model Risk Of Bias Assessment Tool) [73].”
MethodHueting, T. A., van Maaren, M. C., Hendriks, M. P., Koffijberg, H., & Siesling, S. (2023) · The Breast · doi:10.1016/j.breast.2023.04.003 · Cited by 21
“However, the majority of the identified models were found to be at high risk of bias according to the Prediction Model Risk Of Bias Assessment Tool (PROBAST) [6].”
BackgroundThe Sacro-femoral-pubic Angle Is Unreliable to Estimate Pelvic Tilt: A Meta-analysis
Chai, Y., Boudali, A. M., Khadra, S. M., & Walter, W. L. (2023) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000002650 · Cited by 18
“We evaluated the risk of bias using the Prediction Model Risk Of Bias Assessment Tool, which involved low, high, or unclear risk of bias concerns in four domains: participants, predictors, outcome, and analysis [40].”
Prognostic models for surgical-site infection in gastrointestinal surgery: systematic review
McLean, K. A., Goel, T., Lawday, S., Riad, A., Simoes, J. F. F., Knight, S. R., Ghosh, D., Glasbey, J. C., Bhangu, A. A., & Harrison, E. M. (2023) · British journal of surgery · doi:10.1093/bjs/znad187 · Cited by 19
Colunga‐Lozano, L. E., Foroutan, F., Rayner, D. G., Luca, C. D., Hernández‐Wolters, B., Couban, R., Ibrahim, Q. I., & Guyatt, G. H. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.10.016 · Cited by 24
Sadlonova, M., Hansen, N., Esselmann, H., Celano, C. M., Derad, C., Asendorf, T., Chebbok, M., Heinemann, S., Wiesent, A., Schmitz, J., et al. (2023) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2023.12.017 · Cited by 20
Liu, X., Reigle, J. K., Prasath, V. B. S., & Dhaliwal, J. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108093 · Cited by 21
Salehi, M. A., Harandi, H., Mohammadi, S., Farahani, M. S., Shojaei, S., & Saleh, R. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01058-1 · Cited by 19
“In addition, the Prediction Model Study Risk of Bias Assessment Tool (PROBAST) checklist was employed to evaluate the presence of bias and assess the applicability of the included studies [33] (Table E3).”
Ewington, L. J., Black, N., Leeson, C., Wattar, B. H. A., & Quenby, S. M. (2024) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17802 · Cited by 21
“25 This includes assessing the participants, predictors, outcomes and analysis for risk of bias, and then providing an overall risk of bias.”
Clinical performance of automated machine learning: A systematic review
Thirunavukarasu, A. J., Elangovan, K., Gutiérrez, L., Hassan, R., Li, Y., Tan, T. F., Cheng, H., Teo, Z. L., Lim, G., & Ting, D. S. W. (2024) · Annals of the Academy of Medicine Singapore · doi:10.47102/https://doi.org/10.47102/annals-acadmedsg.2023113 · Cited by 21
Stroke risk prediction models: A systematic review and meta-analysis
Asowata, O. J., Okekunle, A. P., Olaiya, M. T., Akinyemi, J. O., Owolabi, M. O., & Akpa, O. M. (2024) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2024.122997 · Cited by 24
Artificial neural networks development in prosthodontics - a systematic mapping review
Bobeică, O., & Iorga, D. (2024) · Journal of Dentistry · doi:10.1016/j.jdent.2024.105385 · Cited by 20
Rehman, A. U., Neyra, J. A., Chen, J., & Ghazi, L. (2025) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2025.2497843 · Cited by 24
“The risk of bias in the included studies was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST) [24].”
Halligan, S., Boone, D. J., Archer, L., Ahmad, T., Bloom, S. L., Rodriguez‐Justo, M., Taylor, S. A., & Mallett, S. (2021) · Health Technology Assessment · doi:10.3310/hta25450 · Cited by 19
“Location of disease l Any location: 10,877; 4193 (32) l Colon only: 8584; 3500 (25) l Colonic any: 70,571; 2999 (23) l Colonic only: surgery OR 0.”
Self-citationA Systematic Literature Review of the Types of Authentication Safety Practices among Internet Users
Kovalan, K., Omar, S. Z., Tang, L., Bolong, J., Abdullah, R., Ghazali, A. H. A., & Pitchan, M. A. (2021) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2021.0120792 · Cited by 18
Helmrich, I. R., Mikolić, A., Kent, D. M., Lingsma, H. F., Wynants, L., Steyerberg, E. W., & Klaveren, D. V. (2022) · Diagnostic and Prognostic Research · doi:10.1186/s41512-022-00122-0 · Cited by 21
“Recently, the Prediction model Risk Of Bias Assessment Tool (PROBAST) tool has become available to assess the risk of bias and concerns regarding applicability of studies that develop and/or validate a multivariable prediction model in systematic reviews [7].”
MethodInfluentialMerkaj, S., Bahar, R. C., Zeevi, T., Lin, M., Ikuta, I., Bousabarah, K., Petersen, G. C., Staib, L. H., Payabvash, S., Mongan, J., et al. (2022) · Cancers · doi:10.3390/cancers14112623 · Cited by 20
“PROBAST-AI is in development too [66].”
MethodInfluentialRisk Prediction Models of Natural Menopause Onset: A Systematic Review
Raeisi‐Dehkordi, H., Kummer, S., Raguindin, P. F., Dejanovic, G. M., Taneri, P. E., Cardona, I., Kastrati, L., Minder, B., Voortman, T., Marques-Vidal, P. M., et al. (2022) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgac461 · Cited by 19
The risk assessment tool for intensive care unit readmission: A systematic review and meta-analysis
Long, J., Wang, M., Li, W., Cheng, J., Yuan, M., Zhong, M., Zhang, Z., & Zhang, C. (2023) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2022.103378 · Cited by 20
Editorial: Opposites Attract at CORR®—Machine Learning and Qualitative Research
Leopold, S. S., Porcher, R., Gebhardt, M. C., Gioe, T. J., Manner, P. A., Rimnac, C. M., & Wongworawat, M. D. (2020) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001466 · Cited by 18
Shojaei, S., Mousavi, A., Kazemian, S., Armani, S., Maleki, S., Fallahtafti, P., Arashlow, F. T., Daryabari, Y., Naderian, M., Alkhouli, M. A., et al. (2025) · Journal of Personalized Medicine · doi:10.3390/jpm15070302 · Cited by 17
“Studies were rated as having a low, high, or unclear risk of bias based on these criteria [16].”
Clinical prediction models using artificial intelligence approaches in dementia
Veronese, N., Bolzetta, F., Gallo, L., Durante, G., Vernuccio, L., Saccaro, C., Gambino, C. M., Custodero, C., Portincasa, P., Morotti, A., et al. (2025) · Aging Clinical and Experimental Research · doi:10.1007/s40520-025-03112-6 · Cited by 21
“The tool also considers the applicability of the model to different populations and settings [21].”
InfluentialStiefler, S., Seibert, K., Domhoff, D., Görres, S., Wolf‐Ostermann, K., & Peschke, D. (2020) · Zeitschrift für Evidenz Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2020.05.001 · Cited by 17
van den Brink, D. A., de Vries, I. S. A., Datema, M., Perot, L., Sommers, R., Daams, J. G., Calis, J. C. J., Brals, D., & Voskuijl, W. P. (2022) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2023.113448 · Cited by 19
Lu, G., Zhang, D., Chen, J. C., Cao, Y., Chai, L., Liu, K., Chong, Z., Zhang, Y., Lu, Y., Heuschen, A., et al. (2023) · Malaria Journal · doi:10.1186/s12936-023-04604-4 · Cited by 20
“Moreover, the adapted Newcastle–Ottawa Scale (aNOS) scale was adapted to appraise the risk of bias in the included studies, as these were mainly observational [24, 25].”
MethodKohn, R. A., Weissman, G. E., Wang, W., Ingraham, N. E., Scott, S., Bayes, B. A., Anesi, G. L., Halpern, S. D., Kipnis, P., Liu, V. X., et al. (2023) · Medical Care · doi:10.1097/mlr.0000000000001878 · Cited by 19
Fink, H. A., Butler, M. E., Claussen, A. M., Collins, E. S., Krohn, K., Taylor, B. C., Tikabo, S. S., Vang, D., Zerzan, N. L., & Ensrud, K. E. (2023) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.4895 · Cited by 17
“Based on the Prediction Model Risk of Bias Assessment Tool (PROBAST), (35,36) two reviewers (MEB, HAF) independently rated eacheligiblestudy for risk ofbias(ROB)for participants,predictors, outcomes (discrimination and calibration), and analysis domains and overall.”
Hua, Y., Yuan, Y., Wang, X., Liu, L., Zhu, J., Li, D., & Tu, P. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1226473 · Cited by 21
“The PROBAST, which was developed by Wolff and his team in 2019, includes a risk of bias assessment and an applicability evaluation (8).”
Challenges of Machine Learning and AI (What Is Next?), Responsible and Ethical AI
Gkontra, P., Quaglio, G., Garmendia, A. T., & Lekadir, K. (2023) · Springer eBooks · doi:10.1007/978-3-031-36678-9_17 · Cited by 22
Zhao, Y., Wang, Y., & Xu, J. (2023) · Academic Radiology · doi:10.1016/j.acra.2023.10.050 · Cited by 17
A systematic review of (semi-)automatic quality control of T1-weighted MRI scans
Hendriks, J., Mutsaerts, H. J. M. M., Joules, R., Peña‐Nogales, Ó., Rodrigues, P., Wolz, R., Burchell, G. L., Barkhof, F., & Schrantee, A. (2023) · Neuroradiology · doi:10.1007/s00234-023-03256-0 · Cited by 17
Development and validation of prediction model for older adults with cognitive frailty
Huang, J., Zeng, X., Ning, H., Peng, R., Guo, Y., Hu, M., & Feng, H. (2024) · Aging Clinical and Experimental Research · doi:10.1007/s40520-023-02647-w · Cited by 17
“According to the Tool to Assess Risk of Bias and Application of Prediction Model Studies (PROBAST) [30], a minimum of 20 events per variable (EPVs) was suggested for model development and at least 100 participants with the outcome for model validation to minimize overfitting.”
MethodMeinke, C., Lueken, U., Walter, H., & Hilbert, K. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105640 · Cited by 18
“The best model of each study was assessed for risk of bias using PROBAST (Prediction model study risk of bias assessment tool; Wolff et al., 2019), a tool developed for predictive modelling in healthcare.”
Mäenpää, S. M., & Korja, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105523 · Cited by 22
Yue, Z., Sun, Y., Pan, Y., Dai, Y., Xiao, Y., & Yu, Y. (2024) · Australian Critical Care · doi:10.1016/j.aucc.2024.05.003 · Cited by 19
He, C., Xu, H., Yuan, E., Ye, L., Chen, Y., Yao, J., & Song, B. (2024) · Insights into Imaging · doi:10.1186/s13244-024-01780-y · Cited by 23
“…the prediction of MIBC and (2) use the Checklist for Arti fi cial Intelligence in Medical Imaging (CLAIM), Radiomics Quality Score (RQS), and Prediction model Risk of Bias Assessment Tool (PROBAST) to comprehensively assess the reporting quality and methodological quality of these models [35 – 37].”
MethodRotbei, S., Tseng, W. H., Barbancho, B. M., Haleem, M. S., Montesinos, L., Pecchia, L., Fico, G., & Alessio Botta, B. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125058 · Cited by 17
Sankar, H., Alagarsamy, R., Lal, B., Rana, S. S., Roychoudhury, A., Agrawal, A., & Wankhar, S. (2024) · Oral Surgery Oral Medicine Oral Pathology and Oral Radiology · doi:10.1016/j.oooo.2024.09.010 · Cited by 17
Risk prediction models for disability in older adults: a systematic review and critical appraisal
Zhou, J., Xu, Y., Yang, D., Zhou, Q., Ding, S., & Pan, H. (2024) · BMC Geriatrics · doi:10.1186/s12877-024-05409-z · Cited by 18
“…(JZ and YX) independently assessed the risk of bias and applicability of included studies using the Prediction Model Risk of Bias Assessment Tool (PROBAST) [31, 32], and a structured five-step approach was used to determine which models were considered clinically valuable for practitioners [33].”
BackgroundMethodMortality prediction models for community-dwelling older adults: A systematic review
Exmann, C. J., Kooijmans, E. C. M., Joling, K. J., Burchell, G. L., Hoogendijk, E. O., & van Hout, H. P. J. (2024) · Ageing Research Reviews · doi:10.1016/j.arr.2024.102525 · Cited by 18
Liu, J., Zhang, H., Woon, D. T. S., Perera, M., & Lawrentschuk, N. (2024) · Cancers · doi:10.3390/cancers16213596 · Cited by 17
Wang, Y., Zhang, Z., Zhang, Z., Chen, X., Liu, J., & Liu, M. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02771-w · Cited by 20
“These methodological issues raise concerns about the reliability and generalisability of the models, while the PROBAST tool published in 2019 [10] has improved bias assessment, its inconsistent application in studies published after 2019 highlights the need for broader adoption of standardised…”
Grodzka, O., Łagowski, W., Eyileten, C., & Domitrz, I. (2025) · The Journal of Headache and Pain · doi:10.1186/s10194-025-02023-1 · Cited by 18
“All studies included in the systematic review were assessed for risk of bias using the Prediction Model Risk of Bias Assessment Tool (PROBAST), a tool for assessing the risk of bias and the applicability of diagnostic and prognostic prediction model studies [43].”
Al-Juhani, A. A., Desoky, R. M., Binshalhoub, A. A., Alzahrani, M. J., Alraythi, M. S., & Alzahrani, F. F. (2025) · Forensic Science Medicine and Pathology · doi:10.1007/s12024-025-01026-3 · Cited by 17
“The reviewers evaluated the methodical quality and applicability of all preclinical predictive studies using the Prediction model Risk Of Bias Assessment Tool (PROBAST) [20].”
What Is Fair? Defining Fairness in Machine Learning for Health
Gao, J., Chou, B., McCaw, Z. R., Thurston, H., Varghese, P., Hong, C., & Gronsbell, J. L. (2025) · Statistics in Medicine · doi:10.1002/sim.70234 · Cited by 30
Oh, B., Hwangbo, S., Jung, T., Min, K., Lee, C., Apio, C., Lee, H., Lee, S., Moon, M. K., Kim, S., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/25852 · Cited by 17
“Wynants et al [34] reviewed 50 COVID-19 prediction models and reported that most of the models have a high risk of bias when evaluated with the prediction model risk of bias assessment tool [37].”
Dudareva, M., Hotchen, A., McNally, M., Hartmann‐Boyce, J., Scarborough, M., & Collins, P. G. S. (2021) · Journal of Bone and Joint Infection · doi:10.5194/jbji-6-257-2021 · Cited by 17
Bryant, A., Hiu, S. K. W., Kunonga, P., Gajjar, K. B., Craig, D., Vale, L. D., Winter-Roach, B. A., Elattar, A., & Naik, R. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015048 · Cited by 19
Risk Prediction Model for Esophageal Cancer Among General Population: A Systematic Review
Chen, R., Zheng, R., Zhou, J., Li, M., Shao, D., Li, X., Wang, S., & Wei, W. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.680967 · Cited by 19
Comparison and ensemble of 2D and 3D approaches for COVID-19 detection in CT images
Ahmed, S. A. A., Yavuz, M. C., Şen, M. U., Gülşen, F., Tutar, O., Korkmazer, B., Samancı, C., Şirolu, S., Hamid, R., Eryurekli, A. E., et al. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.02.018 · Cited by 19
“[35] analyze all COVID-19 AI papers published in the first 9-months period of 2020, in terms of their potential potential biases, according to the criteria indicated in [36].”
Revisión de la validez de las escalas de valoración del riesgo de caídas en pacientes hospitalizados
Gutiérrez‐Valencia, M., Leache, L., & Saiz, L. C. (2022) · Revista Española de Geriatría y Gerontología · doi:10.1016/j.regg.2022.03.005 · Cited by 22
Torres, K. A., Konrade, E. N., White, J., Tavares, M. C. M., Bunch, J. T., Burton, D. L., Jackson, R. S., Birlingmair, J., & Carlson, B. B. (2022) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-022-05763-2 · Cited by 17
“It is organized into 4 domains: participants, predictors, outcome, and analysis [31].”
BackgroundFigueiredo, F. D. A., Ramos, L. E. F., Silva, R. T., Ponce, D., de Carvalho, R. L. R., Schwarzbold, A. V., Maurílio, A. D. O., Scotton, A. L. B. A., Garbini, A. F., Farace, B. L., et al. (2022) · BMC Medicine · doi:10.1186/s12916-022-02503-0 · Cited by 17
“AKI: Acute kidney injury; ARDS: Acute respiratory distress syndrome; AUROC: Area under the receiver operating characteristic; BMI: Body mass index; BPM1: Beats per minute; BPM2: Breaths per minute; BUN: Blood urea nitrogen; CI: Con‑ fidence interval; COPD: Chronic obstructive pulmonary disease; COVID‑19: Cor‑ onavirus…”
Zeng, M., Oakden‐Rayner, L., Bird, A., Smith, L., Wu, Z., Scroop, R., Kleinig, T. J., Jannes, J., Jenkinson, M., & Palmer, L. J. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.945813 · Cited by 18
O’Rielly, C. M., Harrison, T. G., Andruchow, J. E., Ronksley, P. E., Sajobi, T. T., Robertson, H. L., Lorenzetti, D., & McRae, A. D. (2023) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2022.12.028 · Cited by 17
Use of artificial intelligence in paediatric anaesthesia: a systematic review
Antel, R., Sahlas, E., Gore, G. C., & Ingelmo, P. M. (2023) · BJA Open · doi:10.1016/j.bjao.2023.100125 · Cited by 21
“such as the Template for Intervention Description and Replication (TIDieR) checklist and guide for reporting of interventions,(18) and the Prediction Model Study Risk of Bias Assessment Tool (PROBAST) for reporting of prediction models.(20)”
MethodDamen, J. A., & Hooft, L. (2019) · Diagnostic and Prognostic Research · doi:10.1186/s41512-019-0049-6 · Cited by 17
Palazón‐Bru, A., Mares‐García, E., López‐Bru, D., Mares‐Arambul, E., la Rosa, D. M. F.-D., Carbonell‐Torregrosa, M. D. L. Á., & Gil-Guillén, V. F. (2019) · Head & Neck · doi:10.1002/hed.26025 · Cited by 17
Predicción de mortalidad en recién nacidos prematuros. Revisión sistemática actualizada
del Río, R., Thió, M., Bosio, M., Figueras, J., & Iriondo, M. (2020) · Anales de Pediatría · doi:10.1016/j.anpedi.2019.11.003 · Cited by 17
Ethical considerations in the early detection of Alzheimer's disease using speech and AI
Petti, U., Nyrup, R., Skopek, J. M., & Korhonen, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594063 · Cited by 16
Kaul, T., Damen, J. A. A., Wynants, L., Van Calster, B., van Smeden, M., Hooft, L., & Moons, K. G. M. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111732 · Cited by 16
Self-citationChen, H., Han, J. H., Li, J., Xiong, J., Wang, D., Han, M., Shen, Y., & Lu, W. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2024.1522911 · Cited by 18
Dong, Y., Zeng, H., Yang, L., Song, H., & Li, Q. (2025) · Anesthesiology and Perioperative Science · doi:10.1007/s44254-025-00093-7 · Cited by 19
Al-Juhani, A. A., Gaber, A. M., Desoky, R. M., Binshalhoub, A. A., Alzahrani, M. J., Alraythi, M. S., Showail, S., & Aseeri, A. A. (2025) · Forensic Science Medicine and Pathology · doi:10.1007/s12024-025-01002-x · Cited by 16
Zhou, Z., Zhang, J., Fan, C., Wei, Z., Wang, Q., & Liu, C. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2025.1662409 · Cited by 34
Resampling methods for class imbalance in clinical prediction models: A scoping review protocol
Abdelhay, O., Shatnawi, A., Najadat, H., & Altamimi, T. (2025) · PLoS ONE · doi:10.1371/journal.pone.0330050 · Cited by 20
“[34]”
Framework for Assessing Ethical Aspects of Algorithms and Their Encompassing Socio-Technical System
van Bruxvoort, X., & van Keulen, M. (2021) · Applied Sciences · doi:10.3390/app112311187 · Cited by 21
“Methods and tools exist for checking for bias, for example PROBAST [49] which has been effectively used to show that of the 62 algorithms for diagnosing or prognosing COVID-19 passing other quality criteria, only six had a low rick of bias whereas the others either had high (45) or unclear (11)…”
MethodJi, Z., Li, X., Lei, S., Xu, J., & Xie, Y. (2023) · The Clinical Respiratory Journal · doi:10.1111/crj.13606 · Cited by 18
Yu, J., Liu, X., Zhu, Z., Yang, Z., He, J., Zhang, L., & Lu, H. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2023.1138234 · Cited by 18
“We appraised the quality of the included articles by using the Prediction model Risk Of Bias ASsessment Tool (PROBAST), which is an assessment tool developed specifically for diagnostic and prognostic prediction model studies (25).”
MethodDhiman, P., Ma, J., Gibbs, V. N., Rampotas, A., Kamal, H., Arshad, S., Kirtley, S., Doree, C. J., Murphy, M. F., Collins, G. S., et al. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.05.002 · Cited by 17
Zworth, M., Kareemi, H., Boroumand, S., Sikora, L., Stiell, I. G., & Yadav, K. (2023) · Canadian Journal of Emergency Medicine · doi:10.1007/s43678-023-00572-5 · Cited by 17
“Another tool, PROBAST, (Prediction model Risk of Bias Assessment Tool) [29] does include this, but overemphasizes predictor selection for multivariate analysis, which is not relevant to our study question.”
MethodMultivariable prediction models of caries increment: a systematic review and critical appraisal
Havsed, K., Petersson, G. H., Isberg, P., Pigg, M., Svensäter, G., & Rohlin, M. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02298-y · Cited by 17
“Pairs of review authors independently assessed risk of bias (ROB) and concern regarding applicability using PROBAST [5, 6] with 20 signalling questions in 4 domains for ROB (participants, predictors, outcome, analysis) and 3 domains for applicability (participants, predictors, outcome).”
MethodOtten, M., Jagesar, A. R., Dam, T. A., Biesheuvel, L. A., den Hengst, F., Ziesemer, K. A., Thoral, P., de Grooth, H., Girbes, A. R. J., François-Lavet, V., et al. (2023) · Critical Care Medicine · doi:10.1097/ccm.0000000000006100 · Cited by 23
“Therefore, we used the PROBAST tool as source of inspiration for a RL-health-bias-risk-assessment.”
MethodFu, Y., Feller, D., Koes, B. W., & Chiarotto, A. (2024) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2024.12081 · Cited by 17
Validation of the BOADICEA model in a prospective cohort of BRCA1/2 pathogenic variant carriers
Yang, X., Mooij, T. M., Leslie, G., Ficorella, L., Andrieu, N., Kast, K., Singer, C. F., Jakubowska, A., van Gils, C. H., Tan, Y. Y., et al. (2024) · Journal of Medical Genetics · doi:10.1136/jmg-2024-109943 · Cited by 17
“Larger number of carriers at older ages, with a larger number of incident cancers will be required to assess the predicted risks with greater precision, in particular among different risk categories [26].”
BackgroundLv, G., & Wang, Y. (2024) · Technology and Health Care · doi:10.3233/thc-240119 · Cited by 18
Xu, G., Feng, F., Chen, W., Xiao, Y., Fu, Y., Zhou, S., Duan, S., & Li, M. (2024) · Academic Radiology · doi:10.1016/j.acra.2024.07.051 · Cited by 16
Chen, H., Yu, D., Zhang, J., & Li, J. (2024) · Clinical Therapeutics · doi:10.1016/j.clinthera.2024.09.013 · Cited by 18
Post-Cardiac arrest outcome prediction using machine learning: A systematic review and meta-analysis
Zobeiri, A., Rezaee, A., Hajati, F., Argha, A., & Alinejad‐Rokny, H. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105659 · Cited by 23
Vidal, R., Grotle, M., Johnsen, M. B., Yvernay, L., Hartvigsen, J., Ostelo, R., Kjønø, L. G., Enstad, C. L., Killingmo, R. M., Halsnes, E. H., et al. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111593 · Cited by 19
Supervised machine learning for microbiomics: Bridging the gap between current and best practices
Dudek, N. K., Chakhvadze, M., Kobakhidze, S., Kantidze, O. L., & Gankin, Y. V. (2024) · Machine Learning with Applications · doi:10.1016/j.mlwa.2024.100607 · Cited by 20
“Some checklists span both ML and life sciences or healthcare, such as PROBAST 53 .”
Zhu, J., Yang, F., Wang, Y., Wang, Z., Xiao, Y., Wang, L., & Sun, L. (2024) · Journal of Medical Internet Research · doi:10.2196/57641 · Cited by 18
“This approach provides a comprehensive evaluation of the overall risk of bias and overall usability [25].”
Kiya, G. T., Abebe, G., Mekonnen, Z., Tadasa, E., Milkias, G., & Asefa, E. T. (2025) · PLoS ONE · doi:10.1371/journal.pone.0315797 · Cited by 16
Machine learning algorithms for predicting PTSD: a systematic review and meta-analysis
Vali, M., Motahari-Nezhad, H., Kovács, L., & Gandomi, A. H. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02754-2 · Cited by 17
“Our systematic review employs the Prediction model Risk Of Bias ASsessmenT (PROBAST) tool [14], a validated instrument created to evaluate biases in research that develop predictive models.”
Gill, S. S., Ponniah, H. S., Giersztein, S., Anantharaj, R. M., Namireddy, S. R., Killilea, J., Ramsay, D. S. C., Salih, A., Thavarajasingam, A., Scurtu, D., et al. (2025) · Brain and Spine · doi:10.1016/j.bas.2025.104208 · Cited by 20
“Reporting of model performance was inconsistent, with many omitting calibration and clinical utility assessments, including the 95% confidence intervals, despite PROBAST and TRIPOD guidelines, limiting the clinical applicability of their findings (Collins et al., 2021; Wolff et al., 2019).”
InfluentialShin, H. A., Kang, H., & Choi, M. (2025) · Healthcare Informatics Research · doi:10.4258/hir.2025.31.1.23 · Cited by 21
“The risk of bias and applicability of each study were independently assessed by two researchers using the Prediction model Risk of Bias Assessment Tool (PROBAST) checklist [16].”
Islam, S., Shahriyar, R., Agarwala, A., Zaman, M., Ahamed, S. A., Rahman, R., Chowdhury, M. H., Sarker, F., & Mamun, K. A. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-02864-5 · Cited by 19
“Risk of bias and applicability were undertaken using the prediction model Risk of Bias Assessment Tool (PROBAST) [21] and Quadas-2 tools [22] for different types of studies.”
Asadi, F., Rahimi, M., Ramezanghorbani, N., & Almasi, S. (2025) · Cancer Reports · doi:10.1002/cnr2.70138 · Cited by 21
Tariciotti, L., Palmisciano, P., GIORDANO, M., REMOLI, G., LACORTE, E., Bertani, G., Locatelli, M., DIMECO, F., Caccavella, V. M., & PRADA, F. (2021) · Journal of Neurosurgical Sciences · doi:10.23736/s0390-5616.21.05483-7 · Cited by 19
Li, L., Li, X., Li, W., Ding, X., Zhang, Y., Chen, J., & Li, W. (2022) · BMC Cancer · doi:10.1186/s12885-022-09841-5 · Cited by 16
“Moons K, Wolff RF, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration.”
MethodInfluentialWang, S., Huang, H., Hou, M., Xu, Q., Qian, W., Tang, Y., Li, X., Qian, G., Ma, J., Zheng, Y., et al. (2023) · Pediatric Research · doi:10.1038/s41390-023-02558-6 · Cited by 18
“The Prediction Model Risk of Bias Assessment Tool (PROBAST) is useful for assessing prediction model studies and critically appraising prediction model studies.(14,15) It includes 20 signal questions across four domains (Participants, Predictors, Outcomes, and Analysis) that can be used to evaluate the risk of bias (R…”
MethodMaking science computable: Developing code systems for statistics, study design, and risk of bias
Alper, B. S., Dehnbostel, J., Afzal, M., Subbian, V., Soares, A., Kunnamo, I., Shahin, K., & McClure, R. C. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103685 · Cited by 16
Saady, M., Eissa, M., Yacoub, A. S., Hamed, A. B., & Azzazy, H. M. E.-S. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103066 · Cited by 15
Du, M., He, S., Liu, J., & Yuan, L. (2025) · Academic Radiology · doi:10.1016/j.acra.2025.03.054 · Cited by 15
Karimzadhagh, S., Ghodous, S., ROBATI, R. M., Abbaspour, E., Goldust, M., Zaresharifi, N., & Zaresharifi, S. (2025) · International Journal of Dermatology · doi:10.1111/ijd.17981 · Cited by 19
“Meanwhile, ML is particularly adept at handling and analyzing the complexities of healthcare information [18].”
Mohamedahmed, A. Y., Zaman, S., Agrof, M., Adam, M., Husain, N., & Yassin, N. (2025) · International Journal of Surgery · doi:10.1097/js9.0000000000003067 · Cited by 16
“The prediction model Risk of Bias Assessment Tool (PROBAST) was used to evaluate the risk of bias (ROB) [15] .”
Clinical Practice Guideline for Diagnosis and Management of Faltering Weight
Kersten, H. B., Goday, P. S., Abdelhadi, R. A., Adolphe, S., Anania, N., Bennett, D. S., Burke, L. W., Larson‐Nath, C. M., Lee, T. M., Leu, M. G., et al. (2026) · PEDIATRICS · doi:10.1542/peds.2025-075764 · Cited by 18
Metadata
- Authors
- Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group†
- Affiliations
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Utrecht University (NL)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- 51-58
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907554860
- Cited by
- 2,984
- References
- 42
- Citations by year
- 2026: 921 · 2025: 561 · 2024: 416 · 2023: 343 · 2022: 303 · 2021: 250 · 2020: 140 · 2019: 45
- FWCI
- 83.39
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
- Bristol Research (University of Bristol) (acceptedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
- Radboud Repository (Radboud University) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Apollo (University of Cambridge) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Apollo
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- PROBAST, risk of bias assessment, diagnostic model, prognostic model, applicability, critical appraisal, reporting guidelines, Delphi consensus, systematic reviews, personalized prediction
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Delphi Technique, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.