PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · cited 3×
“A domain-based structure was adopted similar to that used in other risk of bias tools such as the revised Cochrane Risk of Bias tool,(29) QUADAS-2,(36) ROBINSI(30) and ROBIS.”
BackgroundMethodSelf-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · cited 3×
“A domain-based structure was adopted similar to that used in other risk of bias tools such as the revised Cochrane Risk of Bias tool,(29) QUADAS-2,(36) ROBINSI(30) and ROBIS.”
InfluentialSelf-citationCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147 · cited 11×
“A multivariable prediction model is defined as any combination or equation of two or more predictors for estimating the probability or risk for an individual.(7, 8, 33-35)”
InfluentialSelf-citation2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.11.005 · Cited by 4,945
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Circulation · doi:10.1161/01.cir.0000437741.48606.98 · Cited by 3,697
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
Self-citationAssessing Bias in Studies of Prognostic Factors
Hayden, J. A., van der Windt, D. A., Cartwright, J. L., Côté, P., & Bombardier, C. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-4-201302190-00009 · Cited by 3,287 · cited 1×
“(25, 26) The QUIPS tool has been developed to assess the risk of bias in predictor finding (prognostic factor) studies.(28) The methodological quality of studies investigating the impact of a prediction model using a comparative randomised design can be assessed using the revised Cochrane risk of bias tool (ROB 2.”
MethodCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
Self-citationSystematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Cited by
Items 401–500 of 1,010
Question answering systems for health professionals at the point of care—a systematic review
Kell, G., Roberts, A., Umansky, S., Qian, L., Ferrari, D., Soboczenski, F., Wallace, B., Patel, N., & Marshall, I. J. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae015 · Cited by 47
Evaluating the impact of the Radiomics Quality Score: a systematic review and meta-analysis
Barry, N., Kendrick, J., Molin, K., Li, S., Rowshanfarzad, P., Hassan, G. M., Dowling, J., Parizel, P. M., Hofman, M. S., & Ebert, M. A. (2025) · European Radiology · doi:10.1007/s00330-024-11341-y · Cited by 40
“…reporting of a multivariable prediction model for individual prognosis or diagnosis [37], PROBAST prediction model risk of bias assessment tool [38], IBSI image biomarker standardisation initiative, CLAIM CheckList for Arti fi cial Intelligence in Medical imaging [48], QUIPS quality in…”
InfluentialThe Limitations of Artificial Intelligence in Head and Neck Oncology
Rao, K. N., Fernandez-Alvarez, V., Guntinas-Lichius, O., Sreeram, M. P., de Bree, R., Kowalski, L. P., Forastiere, A., Pace-Asciak, P., Rodrigo, J. P., Saba, N. F., et al. (2025) · Advances in Therapy · doi:10.1007/s12325-025-03198-4 · Cited by 33
“…standardized protocols for data curation and adherence to reporting frameworks like “Prediction model Risk Of Bias ASsessment Tool” (PROBAST) [60], which is a tool designed to assess the risk of bias in studies that develop or validate predictive models, and “Transparent Reporting of a…”
Liu, L., Li, Z., Hu, Y., Li, C., He, S., Zhang, S., Gao, J., Zhu, H., & Huang, G. (2025) · Journal of Medical Internet Research · doi:10.2196/73052 · Cited by 47
“PROBAST (Prediction Model Risk of Bias Assessment Tool), a standard quality evaluation tool [25], was employed to assess the risk of bias in models from the original studies.”
Shi, T., Lin, Y., Zhao, H., & Kong, G. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf065 · Cited by 35
“27,28 PROBAST consists of 2 main components: Risk of Bias and Applicability.”
Cardiovascular Disease Prognostic Models in Latin America and the Caribbean: A Systematic Review
Carrillo-Larco, R. M., Altez-Fernandez, C., Pacheco-Barrios, N. V., Claudia Bambs, S., Irazola, V. E., Miranda, J. J., Danaei, G., & Perel, P. (2019) · Global Heart · doi:10.1016/j.gheart.2019.03.001 · Cited by 42
“In addition, risk of bias was formally assessed with a validated tool [16-18].”
MethodWirtz, T. H., Buendgens, L., Weiskirchen, R., Loosen, S. H., Haehnsen, N., Puengel, T., Jhaisha, S. A., Brozat, J. F., Hohlstein, P., Koek, G. H., et al. (2020) · Diagnostics · doi:10.3390/diagnostics10110990 · Cited by 33
“Moreover, participants with missing data were omitted from the analysis, resulting in a data analysis bias according to the PROBAST tool [41].”
MethodSmeijers, S., & Depreitere, B. (2021) · European Spine Journal · doi:10.1007/s00586-021-06954-6 · Cited by 33
“* Suggested tabular presentation for PROBAST results [21, 22] PROBAST: Prediction model Risk of Bias Assessment Tool.”
BackgroundMethodInfluentialPolygenic risk prediction models for colorectal cancer: a systematic review
Sassano, M., Mariani, M., Quaranta, G., Pastorino, R., & Boccia, S. (2022) · BMC Cancer · doi:10.1186/s12885-021-09143-2 · Cited by 38
“Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies.”
MethodInfluentialBig data, machine learning, and population health: predicting cognitive outcomes in childhood
Bowe, A. K., Lightbody, G., Staines, A. S., & Murray, D. M. (2022) · Pediatric Research · doi:10.1038/s41390-022-02137-1 · Cited by 38
“58 Prediction model studies aim to develop and validate multivariable prediction models which combine individual predictors to predict an individual’s probability of either having or developing a particular outcome.”
BackgroundPrediction Models for Osteoporotic Fractures Risk: A Systematic Review and Critical Appraisal
Sun, X., Chen, Y., Gao, Y., Zhang, Z., Qin, L., Song, J., Wang, H., & Wu, X. (2022) · Aging and Disease · doi:10.14336/ad.2021.1206 · Cited by 35
“Aging and Disease • Volume 13, Number 4, August 2022 1217 Risk of bias and applicability assessment The risk of bias and applicability of each included study was independently assessed by two reviewers (ZZ and XS) using the prediction model risk of bias assessment tool (PROBAST) [23, 24].”
BackgroundMethodPrediction of depression treatment outcome from multimodal data: a CAN-BIND-1 report
Sajjadian, M., Uher, R., Ho, K. T., Hassel, S., Milev, R. V., Frey, B. N., Farzan, F., Blier, P., Foster, J. A., Parikh, S. V., et al. (2022) · Psychological Medicine · doi:10.1017/s0033291722002124 · Cited by 37
“When designing the development of the prediction model, we followed the current recommendations to reduce the risk of bias (ROB) and overfitting (Moons et al., 2019; Ranstam, Cook, & Collins, 2016; Wolff et al., 2019).”
Prognostic scores in status epilepticus: A systematic review and meta‐analysis
Yuan, F., Damien, C., & Gaspard, N. (2022) · Epilepsia · doi:10.1111/epi.17442 · Cited by 35
An Artificial Intelligence Training Workshop for Diagnostic Radiology Residents
Hu, R., Rizwan, A. P., Hu, Z., Li, A., Chung, A. D., & Kwan, B. Y. M. (2023) · Radiology Artificial Intelligence · doi:10.1148/ryai.220170 · Cited by 42
Cardiovascular risk in patients with type 2 diabetes: A systematic review of prediction models
Galbete, A., Tamayo, I., Librero, J., Enguita-Germán, M., Cambra, K., & Ibáñez-Beroiz, B. (2022) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2021.109089 · Cited by 32
Nneji, G. U., Cai, J., Jianhua, D., Hossin, M. A., Monday, H. N., Oluwasanmi, A., Chikwendu, I. A., Ejiyi, C. J., Ukwuoma, C. C., James, E. C., et al. (2022) · Diagnostics · doi:10.3390/diagnostics12030717 · Cited by 33
“Even though all models performed well, it was suggested in [50] that the probability of bias was extreme in all the articles analyzed by the authors, according to the literature in [51].”
ResultPrognostic Value of Parenteral Nutrition Duration on Risk of Retinopathy of Prematurity
Pivodic, A., Holmström, G., Smith, L. E. H., Hård, A., Löfqvist, C., Al‐Hawasi, A., Larsson, E. K. B., Lundgren, P., Gränse, L., Törnqvist, K., et al. (2023) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2023.2336 · Cited by 32
Tomlinson, E., Cooper, C., Davenport, C. F., Rutjes, A. W., Leeflang, M., Mallett, S., & Whiting, P. F. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111370 · Cited by 42
Self-citationPrognostic clinical prediction models for acute post‐surgical pain in adults: a systematic review
Papadomanolakis‐Pakis, N., Munch, P. V., Carlé, N., Uhrbrand, C. G., Haroutounian, S., & Nikolajsen, L. (2024) · Anaesthesia · doi:10.1111/anae.16429 · Cited by 36
Health Risk Assessment Using Machine Learning: Systematic Review
Abhadiomhen, S. E., Nzeakor, E. O., & Oyibo, K. (2024) · Electronics · doi:10.3390/electronics13224405 · Cited by 37
Artificial Intelligence in Gestational Diabetes Care: A Systematic Review
AlSaad, R., Elhenidy, A. M., Tabassum, A., Odeh, N. B., AboArqoub, E., Odeh, A. E., AlTamimi, M., Abd-Alrazaq, A. A., Thomas, R. M., Bashir, M., et al. (2025) · Journal of Diabetes Science and Technology · doi:10.1177/19322968251355967 · Cited by 37
“20 The modified tool encompassed four key domains tailored to this review: “Participants,” “Index Test” (focused on AI algorithms), “Reference Standard” (representing the ground truth), and “Analysis.””
Kupperman, N., & Hertel, J. N. (2020) · Journal of Athletic Training · doi:10.4085/1062-6050-473-19 · Cited by 34
Ogero, M. O., Sarguta, R. J., Malla, L., Aluvaala, J., Agweyu, A., English, M., Onyango, N. O., & Akech, S. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035045 · Cited by 37
Zhang, K., Zhang, X., Ding, W., Xuan, N., Tian, B., Huang, T., Zhang, Z., Cui, W., Huang, H., & Zhang, G. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.704358 · Cited by 39
Shi, Z., Zhang, Z., Liu, Z., Zhao, L., Ye, Z., Dekker, A. L. A. J., & Wee, L. (2021) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-021-05658-9 · Cited by 34
“Methodological robustness Classical evaluation tools such as Quality in Prognostic Studies (QUIPS) for prognostic studies [16], Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) for diagnostic tests [17], and Prediction model Risk Of Bias ASsessment Tool (PROBAST) [18] were not specifically designed for h…”
MethodMachine Learning in Colorectal Cancer Risk Prediction from Routinely Collected Data: A Review
Burnett, B., Zhou, S., Brophy, S. T., Davies, P., Ellis, P., Kennedy, J., Bandyopadhyay, A., Parker, M., & Lyons, R. A. (2023) · Diagnostics · doi:10.3390/diagnostics13020301 · Cited by 33
“Critical appraisal was based on data assessment with reference to the CHARMS checklist and risk of bias assessment using the PROBAST tool [38,39].”
Gu, Z., Du, Y., Wang, P., Zheng, X., He, J. P., Wang, C., & Zhang, J. (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000695 · Cited by 37
“Similarly, univariate analysis was not implemented as it does not account for multivariate corrections, which may exclude important variables or identify spurious associations [25] .”
Baecker, A., Kim, S., Risch, H. A., Nuckols, T. K., Wu, B. U., Hendifar, A. E., Pandol, S. J., Pisegna, J. R., & Jeon, C. Y. (2019) · PLOS ONE · doi:10.1371/journal.pone.0218580 · Cited by 31
“[29]”
Danielsen, A. S., Franconeri, L., Page, S., Myhre, A. E., Tornes, R. A., Kacelnik, O., & Bjørnholt, J. V. (2023) · BMC Infectious Diseases · doi:10.1186/s12879-023-08182-3 · Cited by 33
Artificial Intelligence and Child Abuse and Neglect: A Systematic Review
Lupariello, F., Sussetto, L., Trani, S. D., & Vella, G. D. (2023) · Children · doi:10.3390/children10101659 · Cited by 37
“Finally, if ≥ 1 domain is judged to be at a high risk of bias, then the PROBAST tool will de fi ne the study as being “with high risk of bias” [10].”
BackgroundMethodInfluentialPredicting amyotrophic lateral sclerosis (ALS) progression with machine learning
Jabbar, M. A. D. A., Guo, L. T., Nag, S., Guo, Y., Simmons, Z., Pioro, E. P., Ramasamy, S., & Yeo, C. J. J. (2023) · Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration · doi:10.1080/21678421.2023.2285443 · Cited by 35
“An analysis of our model by the Prediction model Risk Of Bias ASsessment Tool (PROBAST) (41) showed low risk of bias and low concern of lack of applicability in prediction models (Supplementary Table 3).”
MethodLiu, J., Cundy, T. P., Woon, D. T. S., & Lawrentschuk, N. (2024) · Cancers · doi:10.3390/cancers16030486 · Cited by 32
“The prediction model risk of bias assessment tool (PROBAST) was used to assess the risk of bias (ROB) and the applicability of diagnostic and prognostic prediction model studies [23].”
MethodIndividualized prediction models in ADHD: a systematic review and meta-regression
de Pablo, G. S., Iniesta, R., Bellato, A., Caye, A., Dobrosavljevic, M., Parlatini, V., Garcia‐Argibay, M., Li, L., Cabras, A., Ali, M. H., et al. (2024) · Molecular Psychiatry · doi:10.1038/s41380-024-02606-5 · Cited by 37
“Prediction modelling studies investigate the development of such models, as well as their validation [9].”
BackgroundMethodInfluentialLi, J., Zhu, M., & Li, Y. (2024) · Renal Failure · doi:10.1080/0886022x.2024.2380748 · Cited by 35
“To understand the quality of the current relevant research, we assessed the quality of the included studies using the APPRAISE-AI Tool, which was released on JAMA Network Open in 2023, and the Prediction model Risk Of Bias Assessment Tool (PROBAST) [30,31] (Supplemental Data Tables 2 and 3).”
MethodMohammadzadeh, I., Niroomand, B., Hajikarimloo, B., Habibi, M. A., Mortezaei, A., Behjati, J., Albakr, A. A., & Borghei-Razavi, H. (2025) · Clinical Neurology and Neurosurgery · doi:10.1016/j.clineuro.2025.108762 · Cited by 31
“The risk of bias (ROB) for this study was assessed through the PROBAST tool, which evaluates articles on four aspects, including participants, predictors, results, and analysis [13].”
Machine learning algorithms for heart disease diagnosis: A systematic review
Mao, Y., Jimma, B. L., & Belsty, T. (2025) · Current Problems in Cardiology · doi:10.1016/j.cpcardiol.2025.103082 · Cited by 34
Al‐Ameri, A. A. M., Wei, X., Wen, X., Wei, Q., Guo, H., Zheng, S., & Xu, X. (2020) · Transplant International · doi:10.1111/tri.13585 · Cited by 33
“The quality assessment using the PROBAST tool also showed that only six original risk models had a good quality with low ROB and applicability concerns; however, these models were derived from populations where HCV is the predominant cause of HCC, and there was a lack of independent external validation, with the only…”
MethodChen, X., Lao, Y., Zhang, Y., Qiao, L., & Zhuang, Y. (2020) · Annals of Palliative Medicine · doi:10.21037/apm-20-1183 · Cited by 32
“The risk of bias (ROB) and clinical applicability was assessed by the PROBAST tool (19,20) from 20 questions in four key domains: participants, predictors, outcome, and analysis.”
MethodAsgari, S., Khalili, D., Hosseinpanah, F., & Hadaegh, F. (2021) · Intern J Endocrinol Metab · doi:10.5812/ijem.109206 · Cited by 34
“The prediction studies were critically assessed by the Prediction Model Risk of Bias Assessment tool (PROBAST), which was introduced by Wolff et al. in 2019 (12).”
Does machine learning have a role in the prediction of asthma in children?
Patel, D., Hall, G. L., Broadhurst, D., Smith, A. J., Schultz, A., & Foong, R. E. (2021) · Paediatric Respiratory Reviews · doi:10.1016/j.prrv.2021.06.002 · Cited by 37
Grootven, B. V., Jepma, P., Rijpkema, C., Verweij, L., Leeflang, M., Daams, J. G., Deschodt, M., Milisen, K., Flamaing, J., & Buurman, B. M. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047576 · Cited by 33
Prognostic Models in Severe Traumatic Brain Injury: A Systematic Review and Meta-analysis
Vieira, R. D. C. A., Silveira, J. C. P., Paiva, W. S., de Oliveira, D. V., de Souza, C. P. E., Santos, E. S., & de Sousa, R. M. C. (2022) · Neurocritical Care · doi:10.1007/s12028-022-01547-7 · Cited by 34
“(n = 861) conducted in Europe, Israel, Australia, and the United States Table 2 (continued) Study Model Prediction Model Risk of Bias Assessment Tool (PROBAST) Overall judgement of risk of bias Domain 1: Participants Domain 2: Predictors Domain 3: Outcome Domain 4: Analysis Overall judgement Charry et al 2017 (cohort)…”
BackgroundMethodInfluentialTangel, V. E., Krul, S. D., Stolker, R. J., Bramer, W. M., de Graaff, J. C., & Hoeks, S. E. (2022) · Anesthesiology · doi:10.1097/aln.0000000000004369 · Cited by 32
Monday, H. N., Li, J., Nneji, G. U., Hossin, M. A., Kumar, R., Oluwasanmi, A., James, E. C., Mgbejime, G. T., Umana, E. S., Chikwendu, I. A., et al. (2022) · Diagnostics · doi:10.3390/diagnostics12030741 · Cited by 30
“In general, COVID-19 screening based on CT or CXT images has achieved a significant improvement according to [52].”
BackgroundGreig, L., Aspe, R., Hall, A., Comfort, P., Cooper, K., & Swinton, P. A. (2023) · Sports Medicine · doi:10.1007/s40279-023-01854-9 · Cited by 32
“Inclusion criteria for this review were developed according to the PIRD (Population–Index test–Reference test–Diagnosis of interest) mnemonic [32] detailed in Table 1.”
Deng, W., Wang, D., Wan, Y.-D., Lai, S., Ding, Y., & Wang, X. (2024) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2023.1287434 · Cited by 30
Mathis, M. R., Steffner, K. R., Subramanian, H., Gill, G., Girardi, N. I., Bansal, S., Bartels, K., Khanna, A. K., & Huang, J. (2024) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2024.02.004 · Cited by 33
Oddy, C., Zhang, J., Morley, J., & Ashrafian, H. (2024) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2024-101065 · Cited by 36
“C- statistics were used as the primary outcome for model performance.”
MethodVotto, M., SILVESTRI, A. D., Postiglione, L., Filippo, M. D., Manti, S., Grutta, S. L., Marseglia, G. L., & Licari, A. (2024) · European Respiratory Review · doi:10.1183/16000617.0118-2024 · Cited by 30
“The risk of bias and applicability for eligible studies was assessed according to the prediction model study risk of bias assessment tool (PROBAST), which evaluates the risk of bias and applicability of prediction model studies [10].”
ARTIFICIAL INTELLIGENCE BASED MODELS FOR PREDICTING FOODBORNE PATHOGEN RISK IN PUBLIC HEALTH SYSTEMS
Sarkar, P. R. (2025) · doi:10.63125/7685ne21 · Cited by 31
Liu, L., Blake, V. C., Barman, M., Luxan, B. G., Churches, T., Kennedy, G., Ooi, S., Delaney, G. P., & Jorm, L. R. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf176 · Cited by 57
“The risk of bias of the included studies was assessed using a customized tool, adapted from the Prediction Model Risk of Bias Assessment Tool (PROBAST), 44 modified to reflect NLP-specific considerations.”
van der Gaag, W. H., Chiarotto, A., Heymans, M. W., Enthoven, W. T. M., van Rijckevorsel-Scheele, J., Bierma‐Zeinstra, S., Bohnen, A. M., & Koes, B. W. (2020) · Pain · doi:10.1097/j.pain.0000000000002161 · Cited by 32
“The Prediction model Risk Of Bias ASsessment Tool (PROBAST) was used as an additional checklist to minimize the risk of bias of this study.(36,64) According to the Prognosis Research Strategy (PROGRESS) initiative, this article is part of the first phase of prognostic model research: model development, including inter…”
Beulens, J. W. J., Yauw, J. S., Elders, P. J. M., Feenstra, T., Herings, R. M. C., Slieker, R. C., Moons, K. G. M., Nijpels, G., & van der Heijden, A. A. W. A. (2021) · Diabetologia · doi:10.1007/s00125-021-05448-w · Cited by 35
“Risk of bias and applicability assessment Each article was critically appraised for risk of bias and the models’ applicability to the intended population and setting by two reviewers and by a third reviewer in case of disagreement (JSY, AAH, JWB), using an Excel file based on the Prediction study Risk of Bias Assessme…”
Self-citationAsiimwe, I. G., Blockman, M., Cohen, K., Cupido, C., Hutchinson, C., Jacobson, B. F., Lamorde, M., Morgan, J., Mouton, J. P., Nakagaayi, D., et al. (2021) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12740 · Cited by 30
Alty, J., Bai, Q., Li, R., Lawler, K., George, R. J. S., Hill, E., Bindoff, A. D., Garg, S., Wang, X., Huang, G., et al. (2022) · BMC Neurology · doi:10.1186/s12883-022-02772-5 · Cited by 33
“The PROBAST tool [46] confirmed these combined development and validation strategies lower the risk of bias.”
MethodGao, Y., Li, S., Jin, Y., Zhou, L., Sun, S., Xu, X., Li, S., Yang, H., Zhang, Q., & Wang, Y. (2022) · JMIR Public Health and Surveillance · doi:10.2196/35750 · Cited by 30
“The Prediction Model Risk of Bias Assessment Tool (PROBAST) was used to assess the risk of bias (ROB) and the clinical applicability of the included studies [25,26].”
Geethadevi, G. M., Quinn, T. J., George, J., Anstey, K. J., Bell, J. S., Sarwar, M. R., & Cross, A. J. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014885.pub2 · Cited by 37
The human role to guarantee an ethical AI in healthcare: a five-facts approach
Iniesta, R. (2023) · AI and Ethics · doi:10.1007/s43681-023-00353-x · Cited by 34
“Those developing ML tools are prompted to follow the many guidelines available on good practices in ML models’ development to avoid such methodological issues [99–102].”
(2020) · PubMed · Cited by 45
Dormosh, N., van de Loo, B. P. A., Heymans, M. W., Schut, M. C., Medlock, S. K., van Schoor, N. M., van der Velde, N., & Abu–Hanna, A. (2024) · Age and Ageing · doi:10.1093/ageing/afae131 · Cited by 33
“We used the Prediction study Risk Of Bias Assessment Tool (PROBAST) [18] to evaluate risk of bias and concerns about applicability for models presented as part of the primary analysis.”
MethodMeta-analysis of prediction models for predicting lymph node metastasis in thyroid cancer
Liu, F., Han, F., Lu, L., Chen, Y., Guo, Z., & Yao, J. (2024) · World Journal of Surgical Oncology · doi:10.1186/s12957-024-03566-4 · Cited by 29
“The study assessed the risk of bias and the applicability of the models using PROBAST [39].”
MethodMohammadi, I., Firouzabadi, S. R., Hosseinpour, M., Akhlaghpasand, M., Hajikarimloo, B., Nejad, S. Z., & Nia, P. S. (2024) · BMC Cardiovascular Disorders · doi:10.1186/s12872-024-04336-6 · Cited by 30
Mao, A., Su, J., Ren, M., Chen, S., & Zhang, H. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-05688-0 · Cited by 34
Zhu, Y., Du, M., Li, P., Lu, H., Li, A., & Xu, S. (2025) · International Journal of Implant Dentistry · doi:10.1186/s40729-025-00590-1 · Cited by 37
Mohammadzadeh, I., Hajikarimloo, B., Niroomand, B., Eini, P., Ghanbarnia, R., Habibi, M. A., Albakr, A. A., & Borghei-Razavi, H. (2025) · Neurosurgical Review · doi:10.1007/s10143-025-03419-y · Cited by 33
“The risk of bias and applicability concerns in the included studies were assessed using the PROBAST [ 17] (Prediction Model Risk of Bias Assessment Tool).”
Putting AI at the Centre of Heart Failure Care
Ski, C. F., Thompson, D. R., & Rocca, H. B. (2020) · ESC Heart Failure · doi:10.1002/ehf2.12813 · Cited by 34
“13 , 14 The ability of AI to mimic the human brain and even overcome bias is fast contributing to the conceptualization of personalized precision medicine.”
BackgroundMortality Risk Models for Persons with Dementia: A Systematic Review
Smith, E. E., & Ismail, Z. (2021) · Journal of Alzheimer s Disease · doi:10.3233/jad-201364 · Cited by 30
“Quality was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) [5], following guidance for its use [6].”
Identifying older adults with frailty approaching end-of-life: A systematic review
Hall, A., Boulton, E., Kunonga, P., Spiers, G., Beyer, F. R., Bower, P., Craig, D., Todd, C. J., & Hanratty, B. (2021) · Palliative Medicine · doi:10.1177/02692163211045917 · Cited by 30
Foroutan, F., Guyatt, G. H., Trivella, M., Kreuzberger, N., Skoetz, N., Riley, R. D., Roshanov, P. S., Alba, A. C., Sekercioglu, N., Canelo‐Aybar, C., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.11.024 · Cited by 32
Self-citationHopkins, D., Rickwood, D., Hallford, D. J., & Watsford, C. R. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.945006 · Cited by 33
“Risk of bias (ROB) was assessed by two authors (DH, CW) according to the prediction model study risk of bias assessment tool (PROBAST) developed by Wolff et al. (60).”
MethodCooray, S. D., Boyle, J. A., Soldatos, G., Allotey, J., Wang, H., Fernandez-Felix, B. M., Zamora, J., Thangaratinam, S., & Teede, H. J. (2022) · EClinicalMedicine · doi:10.1016/j.eclinm.2022.101637 · Cited by 34
Risk Stratification for Sarcopenic Obesity in Subjects With Nonalcoholic Fatty Liver Disease
Chun, H. S., Lee, M., Lee, H. A., Lee, S., Kim, S.-Y., Jung, Y. J., Lee, C., Kim, H., Lee, H. A., Kim, H. Y., et al. (2022) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2022.11.031 · Cited by 33
Zhang, H., Wang, Y., Wu, S., Ngo, J., Feng, Y., He, X., Zhang, Y., Wu, X., & Hong, D. (2022) · BMC Nephrology · doi:10.1186/s12882-022-03025-w · Cited by 30
“These differences in outcome determination affect the estimated associations between predictors and outcome and thus the predictive accuracy of the diagnostic models [14].”
BackgroundMethodAminotransferases in Relation to the Severity of Dengue: A Systematic Review
Kalluru, P. K. R., Mamilla, M., Valisekka, S. S., Mandyam, S., Martinez, E. C., Posani, S., Sharma, S., Gopavaram, R. R., Gargi, B., Anvitha, G., et al. (2023) · Cureus · doi:10.7759/cureus.39436 · Cited by 29
“To assess the risk of bias and the applicability of the selected studies, the Prediction Model Risk of Bias Assessment Tool (PROBAST) was employed [7].”
MethodParsaei, M., Taghavizanjani, F., Cattarinussi, G., Moghaddam, H. S., Camillo, F. D., Akhondzadeh, S., Sambataro, F., Brambilla, P., & Delvecchio, G. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.08.034 · Cited by 32
Plate, J. D. J., van de Leur, R. R., Leenen, L. P. H., Hietbrink, F., Peelen, L. M., & Eijkemans, M. J. C. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0847-0 · Cited by 33
Jackson, S. M., Perry, L. A., Caruana, C. B., Ramson, D. M., Campbell, R. J., Liu, Z., Nguyen, J. K., Douglas, N. W. R., Kok, J., & Penny‐Dimri, J. C. (2020) · Vascular and Endovascular Surgery · doi:10.1177/1538574420951315 · Cited by 29
Bourgonje, A. R., van Goor, H., Faber, K. N., & Dijkstra, G. D. H. (2023) · Clinical and Translational Gastroenterology · doi:10.14309/ctg.0000000000000579 · Cited by 31
“, the Prediction model Risk Of Bias ASsessment Tool [PROBAST] (50)).”
Zaka, A., Mutahar, D., Gorcilov, J., Gupta, A. K., Kovoor, J. G., Stretton, B., Mridha, N., Sivagangabalan, G., Thiagalingam, A., Chow, C. K., et al. (2024) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztae074 · Cited by 32
Liawrungrueang, W., Cholamjiak, W., Promsri, A., Jitpakdee, K., Sunpaweravong, S., Kotheeranurak, V., & Sarasombath, P. (2025) · Global Spine Journal · doi:10.1177/21925682251314379 · Cited by 28
A systematic review of passive data for remote monitoring in psychosis and schizophrenia
Bladon, S., Eisner, E., Bucci, S., Oluwatayo, A., Martin, G. P., Sperrin, M., Ainsworth, J. D., & Faulkner, S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01451-2 · Cited by 37
“ThesestudieswerealsoassessedforqualityusingeithertheQualityin PrognosisTools(QUIPS) 103 or the Prediction model Risk Of Bias Assessment Tool (PROBAST) 104 for prognostic factor and prediction model studies, respectively.”
Carrillo-Larco, R. M., Aparcana-Granda, D. J., Mejia, J. R., Barengo, N. C., & Bernabé‐Ortiz, A. (2019) · Diabetic Medicine · doi:10.1111/dme.14114 · Cited by 32
“Risk of bias was assessed following the PROBAST recommendations, a risk of bias assessment tool developed exclusively for prognosis models [21,22].”
Validation of Risk Prediction Models to Detect Asymptomatic Carotid Stenosis
Poorthuis, M. H. F., Halliday, A. W., Massa, M. S., Sherliker, P., Clack, R. M., Morris, D., Clarke, R., de Borst, G. J., Bulbulia, R. A., & Lewington, S. (2020) · Journal of the American Heart Association · doi:10.1161/jaha.119.014766 · Cited by 30
Moekotte, A. L., van Roessel, S. R., Malleo, G., Rajak, R., Ecker, B. L., Fontana, M., Han, H., Rabie, M., Roberts, K. J., Khalil, K., et al. (2020) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2020.04.011 · Cited by 32
Stephens, M. E., O’Neal, C. M., Westrup, A. M., Muhammad, F., McKenzie, D. M., Fagg, A. H., & Smith, Z. A. (2021) · Neurosurgical Review · doi:10.1007/s10143-021-01624-z · Cited by 30
Nadarajah, R., Alsaeed, E. S., Hurdus, B., Aktaa, S., Hogg, D., Bates, M. G. D., Cowan, C., Wu, J., & Gale, C. P. (2021) · Heart · doi:10.1136/heartjnl-2021-320036 · Cited by 31
Huang, J., Shlobin, N. A., DeCuypere, M. G., & Lam, S. K. (2021) · Neurosurgery · doi:10.1227/neu.0000000000001736 · Cited by 32
Prediction models in first-episode psychosis: systematic review and critical appraisal
Lee, R., Leighton, S., Thomas, L., Gkoutos, G. V., Wood, S. J., Fenton, S., Deligianni, F., Cavanagh, J., & Mallikarjun, P. K. (2022) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.219 · Cited by 32
Structured reporting to improve transparency of analyses in prognostic marker studies
Sauerbrei, W., Haeussler, T., Balmford, J., & Huebner, M. (2022) · BMC Medicine · doi:10.1186/s12916-022-02304-5 · Cited by 31
“BCRT : Breast Cancer Research and Treatment; CHARMS: Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies; CI: Confidence interval; CONSORT: Consolidated Standards of Reporting Trials; E&E: Explanation and elaboration; EJC: European Journal of Cancer; HR: Hazard r…”
MethodWheatley, J., Liu, Z., Loth, J., Plummer, M. P., Penny‐Dimri, J. C., Segal, R., Smith, J. A., & Perry, L. A. (2022) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.14170 · Cited by 31
Sheehy, J. J., Rutledge, H., Acharya, U. R., Loh, H. W., Gururajan, R., Tao, X., Zhou, X., Li, Y., Gurney, T., & Kondalsamy‐Chennakesavan, S. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102536 · Cited by 32
Sterner, P., Goretzko, D., & Pargent, F. (2023) · Psychological Methods · doi:10.1037/met0000586 · Cited by 31
Liu, P., Sun, Y. H., Zhao, X., & Yan, Y. (2023) · BioMedical Engineering OnLine · doi:10.1186/s12938-023-01159-y · Cited by 31
“For risk of bias, the Prediction Model Risk of Bias Assessment Tool (PROBAST), which focuses on methodological evaluation, is selected [25], and PROBAST is a risk of bias assessment tool for predictive model studies published by the Cochrane Assist Group in 2019.”
MethodAI/ML advances in non-small cell lung cancer biomarker discovery
Çalışkan, M., & Tazaki, K. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1260374 · Cited by 30
White, H. J., Bradley, J., Hadgis, N., Wittke, E., Piland, B., Tuttle, B., Erickson, M. M., & Horn, M. E. (2020) · Current Reviews in Musculoskeletal Medicine · doi:10.1007/s12178-020-09630-2 · Cited by 30
for the IPPIC Collaborative Network, Snell, K. I. E., Allotey, J., Smuk, M., Hooper, R., Chan, C., Ahmed, A. S., Chappell, L. C., von Dadelszen, P., Green, M., et al. (2020) · BMC Medicine · doi:10.1186/s12916-020-01766-9 · Cited by 33
“Two independent reviewers assessed the quality of each IPD cohort using a modified version of the PROBAST (Prediction study Risk of Bias Assessment) tool [16].”
MethodSelf-citationSong, Z., Yang, Z., Hou, M., & Shi, X. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.951881 · Cited by 34
“We used the prediction model risk of bias assessment tool (PROBAST) (11) and the external prognostic validity model to assess the risk of bias in the included studies.”
InfluentialArtificial Intelligence in Lymphoma Histopathology: Systematic Review
Fu, Y., Huang, Z., Deng, X., Xu, L., Liu, Y., Zhang, M., Liu, J., & Huang, B. (2025) · Journal of Medical Internet Research · doi:10.2196/62851 · Cited by 32
“The risk of bias in the models of interest was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST) [10].”
Can Large Language Models Serve as Reliable Tools for Information in Dentistry? A Systematic Review
Alhazmi, N., Alshehri, A., BaHammam, F. A., Philip, M. R., Nadeem, M., & Khanagar, S. B. (2025) · International Dental Journal · doi:10.1016/j.identj.2025.04.015 · Cited by 30
Artificial Intelligence for Noninvasive Health Diagnostics
Wankhede, P., Bhuyar, D. L., Zanwar, S. R., Pawar, R., Jadhav, M. R., Gandhewar, N., Kulkarni, M. B., Bhaiyya, M. L., & Haick, H. (2025) · ACS Sensors · doi:10.1021/acssensors.5c03171 · Cited by 32
Metadata
- Authors
- Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group†
- Affiliations
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Utrecht University (NL)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- 51-58
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907554860
- Cited by
- 2,984
- References
- 42
- Citations by year
- 2026: 921 · 2025: 561 · 2024: 416 · 2023: 343 · 2022: 303 · 2021: 250 · 2020: 140 · 2019: 45
- FWCI
- 83.39
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
- Bristol Research (University of Bristol) (acceptedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
- Radboud Repository (Radboud University) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Apollo (University of Cambridge) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Apollo
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- PROBAST, risk of bias assessment, diagnostic model, prognostic model, applicability, critical appraisal, reporting guidelines, Delphi consensus, systematic reviews, personalized prediction
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Delphi Technique, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.