PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration
Open accessThe DOI matches its Crossref record; 194 of the 201 references it cites were found in a registry or an index.
Abstract
Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMeasuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Research works printed but not linked: 2
- 10.1177/0962280218785504.30032705 · doi:10.1177/0962280218785504.30032705
- Wong SS, 2003, AMIA Annu Symp Proc, 728
Cited by
Items 201–300 of 1,003
Araújo, A. L. D., Moraes, M. C., Pérez-de-Oliveira, M. E., Silva, V. M. D., Saldivia-Siracusa, C., Pedroso, C. M., Lopes, M. A., Vargas, P. A., Kochanny, S., Pearson, A., et al. (2023) · Oral Oncology · doi:10.1016/j.oraloncology.2023.106386 · Cited by 36
Reeve, K. A., Ön, B. I., Havla, J. B., Burns, J., Gosteli, M., Alabsawi, A., Alayash, Z., Götschi, A., Seibold, H., Mansmann, U., et al. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013606.pub2 · Cited by 59
Prediction models for postoperative pulmonary complications: a systematic review and meta-analysis
Huang, Z., Han, Y., Zhuang, H., Jiang, J., Zhou, C., & Yu, H. (2025) · British Journal of Anaesthesia · doi:10.1016/j.bja.2025.04.025 · Cited by 48
Mari, T., Henderson, J., Maden, M., Nevitt, S. J., Duarte, R. V., & Fallon, N. B. (2021) · Journal of Pain · doi:10.1016/j.jpain.2021.07.011 · Cited by 41
Challenges in translational machine learning
Couckuyt, A., Seurinck, R., Emmaneel, A., Quintelier, K. L. A., Novák, D., Gassen, S. V., & Saeys, Y. (2022) · Human Genetics · doi:10.1007/s00439-022-02439-8 · Cited by 42
Foryciarz, A., Pfohl, S. R., Patel, B., & Shah, N. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100460 · Cited by 36
External validation of VO2max prediction models based on recreational and elite endurance athletes
Wiecha, S., Kasiak, P. S., Cieśliński, I., Takken, T., Pałka, T., Knechtle, B., Nikolaidis, P. Τ., Małek, Ł. A., Postula, M. A., Mamcarz, A., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280897 · Cited by 48
“At the same time, we emphasize that there is a significant need to create new, more advanced models under unified guidelines and with the incorporation of PROBAST-AI [60] and TRIPOD checklist [35].”
MethodThe Revised Baux Score as a Predictor of Burn Mortality: A Systematic Review and Meta-Analysis
Edgar, M. C., Bond, S. M., Jiang, S. H., Scharf, I. M., Bejarano, G., & Vrouwe, S. Q. (2023) · Journal of Burn Care & Research · doi:10.1093/jbcr/irad075 · Cited by 35
Ali, I. E., Tanikawa, C., Chikai, M., Ino, S., Sumita, Y. I., & Wakabayashi, N. (2023) · Journal of Prosthodontic Research · doi:10.2186/jpr.jpr_d_23_00073 · Cited by 49
Prognostic models for amyotrophic lateral sclerosis: a systematic review
Xu, L., He, B., Zhang, Y., Chen, L., Fan, D., Zhan, S., & Wang, S. (2021) · Journal of Neurology · doi:10.1007/s00415-021-10508-7 · Cited by 35
Chowdhury, M., Cervantes, E. G., Chan, W.-Y., & Seitz, D. P. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.738466 · Cited by 46
Machine Learning Models for Early Prediction of Sepsis on Large Healthcare Datasets
Camacho-Cogollo, J. E., Bonet, I., Gil, B., & Iadanza, E. (2022) · Electronics · doi:10.3390/electronics11091507 · Cited by 44
Amacher, S. A., Blatter, R., Briel, M., Appenzeller‐Herzog, C., Bohren, C., Becker, C., Beck, K., Groß, S., Tisljar, K., Sutter, R., et al. (2022) · Critical Care · doi:10.1186/s13054-022-04263-y · Cited by 36
The role of artificial intelligence in implant dentistry: a systematic review
Vázquez-Sebrango, G., Anitua, E., Macía, I., & Arganda‐Carreras, I. (2025) · International Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.ijom.2025.04.005 · Cited by 46
Cardiovascular Disease Prognostic Models in Latin America and the Caribbean: A Systematic Review
Carrillo-Larco, R. M., Altez-Fernandez, C., Pacheco-Barrios, N. V., Claudia Bambs, S., Irazola, V. E., Miranda, J. J., Danaei, G., & Perel, P. (2019) · Global Heart · doi:10.1016/j.gheart.2019.03.001 · Cited by 42
Janka, E. A., Várvölgyi, T., Sipos, Z., Soós, A., Hegyi, P. J., Kiss, S., Dembrovszky, F., Csupor, D., Kéringer, P., Pécsi, D., et al. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2021.772165 · Cited by 40
Jenkins, L. C., Chang, W., Buscemi, V., Liston, M. B., Humburg, P., Nicholas, M. K., Graven‐Nielsen, T., Hodges, P. W., McAuley, J. H., & Schabrun, S. M. (2022) · Pain · doi:10.1097/j.pain.0000000000002684 · Cited by 39
Moes, H. R., Henriksen, T., Sławek, J. W., Phokaewvarangkul, O., Buskens, E., & van Laar, T. (2023) · Journal of Neural Transmission · doi:10.1007/s00702-023-02656-z · Cited by 39
“One possible method to determine the risk of bias in prediction model studies is PROBAST (Prediction model Risk of Bias Assessment Tool) (Moons et al. 2019).”
MethodMachine learning models for predicting pre-eclampsia: a systematic review protocol
Ranjbar, A., Taeidi, E., Mehrnoush, V., Roozbeh, N., & Darsareh, F. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-074705 · Cited by 42
Cardiovascular disease risk models and dementia or cognitive decline: a systematic review
Jia, R., Wang, Q., Huang, H., Yang, Y., Chung, Y. F., & Liang, T. (2023) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2023.1257367 · Cited by 39
Moglia, V., Johnson, O. A., Cook, G., de Kamps, M., & Smith, L. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02473-w · Cited by 45
“…sample size; (3) input data, including data type and fields used; (4) ML meth-ods, including any feature engineering, the models used, evaluation metrics, and validation methods; (5) reproducibility, including data and code availability; and (6) quality assessment using the PROBAST framework [22].”
Boakye, N. F., O’Toole, C. C., Jalali, A., & Hannigan, A. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105887 · Cited by 47
van Roessel, S. R., Janssen, B. V., Soer, E. C., Sarasqueta, A. F., Verbeke, C. S., Luchini, C., Brosens, L. A. A., Verheij, J., & Besselink, M. G. (2020) · British journal of surgery · doi:10.1093/bjs/znaa031 · Cited by 34
Machine Learning Models for Image-Based Diagnosis and Prognosis of COVID-19: Systematic Review
Montazeri, M., ZahediNasab, R., Farahani, A., Mohseni, H., & Ghasemian, F. (2021) · JMIR Medical Informatics · doi:10.2196/25181 · Cited by 34
Smeijers, S., & Depreitere, B. (2021) · European Spine Journal · doi:10.1007/s00586-021-06954-6 · Cited by 33
MRI Radiomics and Predictive Models in Assessing Ischemic Stroke Outcome—A Systematic Review
Dragoș, H.-M., Stan, A. D., Pintican, R. M., Feier, D. S., Lebovici, A., Panaitescu, P.-Ș., Dina, C., Strilciuc, Ș., & Muresanu, D. F. (2023) · Diagnostics · doi:10.3390/diagnostics13050857 · Cited by 37
Machine learning‐based prediction models for pressure injury: A systematic review and meta‐analysis
Pei, J., Guo, X., Tao, H., Wei, Y., Zhang, H., Ma, Y., & Han, L. (2023) · International Wound Journal · doi:10.1111/iwj.14280 · Cited by 41
Abdulazeem, H. M., Whitelaw, S., Schauberger, G., & Klug, S. J. (2023) · PLoS ONE · doi:10.1371/journal.pone.0274276 · Cited by 46
Hajishah, H., Kazemi, D., Safaee, E., Amini, M. J., Peisepar, M., Tanhapour, M. M., & Tavasol, A. (2025) · BMC Cardiovascular Disorders · doi:10.1186/s12872-025-04700-0 · Cited by 46
“To evaluate the risk of bias of the included models, PRO-BAST assessment tool was utilized [10].”
Systematic review of clinical prediction models to support the diagnosis of asthma in primary care
Daines, L., McLean, S., Buelo, A., Lewis, S. C., Sheikh, A., & Pinnock, H. (2019) · npj Primary Care Respiratory Medicine · doi:10.1038/s41533-019-0132-z · Cited by 38
Ogero, M. O., Sarguta, R. J., Malla, L., Aluvaala, J., Agweyu, A., English, M., Onyango, N. O., & Akech, S. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035045 · Cited by 37
Chen, X., Lao, Y., Zhang, Y., Qiao, L., & Zhuang, Y. (2020) · Annals of Palliative Medicine · doi:10.21037/apm-20-1183 · Cited by 32
Prognostic Models Predicting Mortality in Preterm Infants: Systematic Review and Meta-analysis
van Beek, P. E., Andriessen, P., Onland, W., & Schuit, E. (2021) · PEDIATRICS · doi:10.1542/peds.2020-020461 · Cited by 34
Buchlak, Q. D., Esmaili, N., Bennett, C. C., & Farrokhi, F. R. (2021) · Acta neurochirurgica. Supplementum · doi:10.1007/978-3-030-85292-4_32 · Cited by 39
Prediction of depression treatment outcome from multimodal data: a CAN-BIND-1 report
Sajjadian, M., Uher, R., Ho, K. T., Hassel, S., Milev, R. V., Frey, B. N., Farzan, F., Blier, P., Foster, J. A., Parikh, S. V., et al. (2022) · Psychological Medicine · doi:10.1017/s0033291722002124 · Cited by 37
Adeoye, J., Hui, L., & Su, Y. (2023) · Journal Of Big Data · doi:10.1186/s40537-023-00703-w · Cited by 40
“Quality rating of the selected studies was performed using the Prediction model Risk of Bias Assessment tool (PROBAST) [32] with domains evaluating the study participants, predictors, outcome, and analytical methods.”
BackgroundMethodPrediction Models for Osteoporotic Fractures Risk: A Systematic Review and Critical Appraisal
Sun, X., Chen, Y., Gao, Y., Zhang, Z., Qin, L., Song, J., Wang, H., & Wu, X. (2022) · Aging and Disease · doi:10.14336/ad.2021.1206 · Cited by 35
Yang, Q., Fan, X., Cao, X. Q., Hao, W., Lu, J., Wei, J., Tian, J., Yin, M., & Ge, L. (2022) · Acta Obstetricia Et Gynecologica Scandinavica · doi:10.1111/aogs.14475 · Cited by 38
“26 In addition, selection of predictors based on univariable analysis was one cause of high risk of bias because the method could lead to incorrect predictor selection.”
BackgroundMethodInfluentialGao, Y., Li, S., Jin, Y., Zhou, L., Sun, S., Xu, X., Li, S., Yang, H., Zhang, Q., & Wang, Y. (2022) · JMIR Public Health and Surveillance · doi:10.2196/35750 · Cited by 30
Tangel, V. E., Krul, S. D., Stolker, R. J., Bramer, W. M., de Graaff, J. C., & Hoeks, S. E. (2022) · Anesthesiology · doi:10.1097/aln.0000000000004369 · Cited by 32
Yuan, T., Edelmann, D., Fan, Z., Alwers, E., Kather, J. N., Brenner, H., & Hoffmeister, M. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102589 · Cited by 34
Prognostic Value of Parenteral Nutrition Duration on Risk of Retinopathy of Prematurity
Pivodic, A., Holmström, G., Smith, L. E. H., Hård, A., Löfqvist, C., Al‐Hawasi, A., Larsson, E. K. B., Lundgren, P., Gränse, L., Törnqvist, K., et al. (2023) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2023.2336 · Cited by 32
A systematic review on passive sensing for the prediction of suicidal thoughts and behaviors
Büscher, R., Winkler, T., Mocellin, J., Homan, S., Josifovski, N., Čihařová, M., van Breda, W., Kwon, S., Larsen, M. E., Torous, J. B., et al. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00089-4 · Cited by 62
“In addition, studies should avoid choosing arbitrary cut-offs for differentiating participants into high-and low-risk 24 .”
BackgroundMethodInfluentialXu, Z., Fan, L., Xin, Y., Wang, Y., & Wang, Y. (2024) · Respiratory Research · doi:10.1186/s12931-024-03033-4 · Cited by 42
Validation of Risk Prediction Models to Detect Asymptomatic Carotid Stenosis
Poorthuis, M. H. F., Halliday, A. W., Massa, M. S., Sherliker, P., Clack, R. M., Morris, D., Clarke, R., de Borst, G. J., Bulbulia, R. A., & Lewington, S. (2020) · Journal of the American Heart Association · doi:10.1161/jaha.119.014766 · Cited by 30
Moekotte, A. L., van Roessel, S. R., Malleo, G., Rajak, R., Ecker, B. L., Fontana, M., Han, H., Rabie, M., Roberts, K. J., Khalil, K., et al. (2020) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2020.04.011 · Cited by 32
van der Gaag, W. H., Chiarotto, A., Heymans, M. W., Enthoven, W. T. M., van Rijckevorsel-Scheele, J., Bierma‐Zeinstra, S., Bohnen, A. M., & Koes, B. W. (2020) · Pain · doi:10.1097/j.pain.0000000000002161 · Cited by 32
Systematic review and meta-analysis of mortality risk prediction models in adult cardiac surgery
Sinha, S., Dimagli, A., Dixon, L. K., Gaudino, M. F. L., Caputo, M., Vohra, H. A., Angelini, G. D., & Benedetto, U. (2021) · Interactive Cardiovascular and Thoracic Surgery · doi:10.1093/icvts/ivab151 · Cited by 34
Salem, H., Soria, D., Lund, J. N., & Awwad, A. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01585-9 · Cited by 38
Evaluation of Prognostic and Predictive Models in the Oncology Clinic
Craddock, M., Crockett, C., McWilliam, A., Price, G. J., Sperrin, M., van der Veer, S. N., & Finn, C. F. (2021) · Clinical Oncology · doi:10.1016/j.clon.2021.11.022 · Cited by 35
Geethadevi, G. M., Quinn, T. J., George, J., Anstey, K. J., Bell, J. S., Sarwar, M. R., & Cross, A. J. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014885.pub2 · Cited by 37
Gokhale, S., Taylor, D., Gill, J., Hu, Y., Zeps, N., Lequertier, V., Prado, L., Teede, H. J., & Enticott, J. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1192969 · Cited by 48
“Using the PROBAST tool (30), studies were rated as low/moderate/high concern for bias and applicability in each of the four domains: participants, predictors, outcomes, and analysis (23, 29).”
BackgroundMethodPredicting risk of postpartum hemorrhage using machine learning approach: A systematic review
Ranjbar, A., Ghamsari, S. R., Boujarzadeh, B., Mehrnoush, V., & Darsareh, F. (2023) · Gynecology and Obstetrics Clinical Medicine · doi:10.1016/j.gocm.2023.07.002 · Cited by 29
Dijkstra, H., van de Kuit, A., de Groot, T. M., Canta, O., Groot, O. Q., Oosterhoff, J. H. F., Doornberg, J. N., van den Bekerom, M., Calderon, S. L., Colaris, J., et al. (2024) · Bone & Joint Open · doi:10.1302/2633-1462.51.bjo-2023-0095.r1 · Cited by 32
“12,63,64 Many studies chose to remove patients with missing data from the analysis (complete case analysis).”
BackgroundMethodImplementation and Updating of Clinical Prediction Models: A Systematic Review
Saelmans, A., Seinen, T. M., Pera, V., Markus, A. F., Fridgeirsson, E. A., John, L. H., Schiphof‐Godart, L., Rijnbeek, P. R., Reps, J., & Williams, R. D. (2025) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2025.100228 · Cited by 53
“23 This fi nding is partially reinforced by Andaur Navarro et al. 61 However, an argument for using observational health data is that this is often the domain in which these models are applied.”
Jia, K. M., Eilerts-Spinelli, H., Edun, O., Lam, K., Howes, A., Thomas, M. L., & Eaton, J. W. (2022) · Journal of the International AIDS Society · doi:10.1002/jia2.25861 · Cited by 29
Zhuo, X.-Y., Lei, S.-H., Sun, L., Bai, Y., Wu, J., Zheng, Y.-J., Liu, K., Liu, W.-F., & Zhao, B.-C. (2024) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.02.018 · Cited by 29
Triggs, T., Crawford, K., Hong, J. G. S., Clifton, V. L., & Kumar, S. (2024) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2024.101054 · Cited by 42
van Boekel, A. M., van der Meijden, S. L., Arbous, M. S., Nelissen, R. G. H. H., Veldkamp, K. E., Nieswaag, E. B., Jochems, K. F. T., Holtz, J., Veenstra, A. V. I., Reijman, J., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0312968 · Cited by 36
Brand, E. C., Elias, S. G., Minderhoud, I. M., van der Veen, J. J., Baert, F. J., Laharie, D., Bossuyt, P. J., Bouhnik, Y., Buisson, A., Lambrecht, G. L. Y., et al. (2019) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2019.12.014 · Cited by 30
Caruana, C. B., Jackson, S. M., Khuong, J. N., Campbell, R. J., Liu, Z., Ramson, D. M., Douglas, N. W. R., Kok, J., Perry, L. A., & Penny‐Dimri, J. C. (2020) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2020.03.039 · Cited by 31
Mortality Risk Models for Persons with Dementia: A Systematic Review
Smith, E. E., & Ismail, Z. (2021) · Journal of Alzheimer s Disease · doi:10.3233/jad-201364 · Cited by 30
Systematic review: non‐invasive prognostic tests for primary sclerosing cholangitis
Mazhar, A., & Russo, M. W. (2021) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.16296 · Cited by 30
Hopkins, D., Rickwood, D., Hallford, D. J., & Watsford, C. R. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.945006 · Cited by 33
Santos, R. J., Coleman, H. G., Cairnduff, V., & Kunzmann, A. T. (2022) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000002022 · Cited by 33
Machine Learning in Colorectal Cancer Risk Prediction from Routinely Collected Data: A Review
Burnett, B., Zhou, S., Brophy, S. T., Davies, P., Ellis, P., Kennedy, J., Bandyopadhyay, A., Parker, M., & Lyons, R. A. (2023) · Diagnostics · doi:10.3390/diagnostics13020301 · Cited by 33
“Critical appraisal was based on data assessment with reference to the CHARMS checklist and risk of bias assessment using the PROBAST tool [38,39].”
MethodZu, W., Huang, X., Xu, T., Du, L., Wang, Y., Wang, L., & Nie, W. (2023) · PLoS ONE · doi:10.1371/journal.pone.0287308 · Cited by 40
“PLOS ONE | https://doi.org/10.1371/journal.pone.0287308 June 28, 2023 3 / 14 PROBAST (Prediction model Risk Of Bias ASsessment Tool) was used to guide the risk of bias assessment in this review [15].”
BackgroundInfluentialA scoping review of policies related to reducing energy drink consumption in children
Rostami, M., Babashahi, M., Ramezani, S., & Dastgerdizad, H. (2024) · BMC Public Health · doi:10.1186/s12889-024-19724-y · Cited by 41
Carrillo-Larco, R. M., Aparcana-Granda, D. J., Mejia, J. R., Barengo, N. C., & Bernabé‐Ortiz, A. (2019) · Diabetic Medicine · doi:10.1111/dme.14114 · Cited by 32
Buchan, T. A., Malik, A., Chan, C., Chambers, J., Suk, Y., Zhu, J. W., Ge, F. Z., Huang, L. M., Vargas, L. A., Hao, Q., et al. (2021) · Heart · doi:10.1136/heartjnl-2021-319243 · Cited by 27
Beulens, J. W. J., Yauw, J. S., Elders, P. J. M., Feenstra, T., Herings, R. M. C., Slieker, R. C., Moons, K. G. M., Nijpels, G., & van der Heijden, A. A. W. A. (2021) · Diabetologia · doi:10.1007/s00125-021-05448-w · Cited by 35
Self-citationDoes machine learning have a role in the prediction of asthma in children?
Patel, D., Hall, G. L., Broadhurst, D., Smith, A. J., Schultz, A., & Foong, R. E. (2021) · Paediatric Respiratory Reviews · doi:10.1016/j.prrv.2021.06.002 · Cited by 37
Fernandez-Felix, B. M., Barca, L. V., García‐Esquinas, E., Correa-Pérez, A., Fernández‐Hidalgo, N., Muriel, A., López‐Alcalde, J., Díaz, N. Á., Pijoán, J. I., Ribera, A., et al. (2021) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2021.05.051 · Cited by 29
Appropriate use of machine learning in healthcare
Ozaydin, B., Berner, E. S., & Cimino, J. J. (2021) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2021.100041 · Cited by 33
Foroutan, F., Guyatt, G. H., Trivella, M., Kreuzberger, N., Skoetz, N., Riley, R. D., Roshanov, P. S., Alba, A. C., Sekercioglu, N., Canelo‐Aybar, C., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.11.024 · Cited by 32
Self-citationSheehy, J. J., Rutledge, H., Acharya, U. R., Loh, H. W., Gururajan, R., Tao, X., Zhou, X., Li, Y., Gurney, T., & Kondalsamy‐Chennakesavan, S. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102536 · Cited by 34
White, H. J., Bradley, J., Hadgis, N., Wittke, E., Piland, B., Tuttle, B., Erickson, M. M., & Horn, M. E. (2020) · Current Reviews in Musculoskeletal Medicine · doi:10.1007/s12178-020-09630-2 · Cited by 30
Prediction models for breast cancer-related lymphedema: a systematic review and critical appraisal
Qiu, L., Yang, T., Jin, Y., & Die, Y. M. (2022) · Systematic Reviews · doi:10.1186/s13643-022-02084-2 · Cited by 32
“Abbreviations BCRL: Breast cancer-related lymphedema; PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses; CHARMS: Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies; PROBAST: Prediction model Risk Of Bias Assessment Tool; ROC: Receiver op…”
BackgroundMethodInfluentialHong, J. G. S., Crawford, K., Jarrett, K., Triggs, T., & Kumar, S. (2024) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2024.101011 · Cited by 37
Risk Prediction Models for Oral Cancer: A Systematic Review
Espressivo, A., Pan, Z., Usher‐Smith, J. A., & Harrison, H. (2024) · Cancers · doi:10.3390/cancers16030617 · Cited by 27
“…Prognosis or Diagnosis (TRIPOD) guidelines [30,31] and we assessed both their risk of bias and applicability to a risk-stratified screening programme using the Prediction Model Risk of Bias Assessment Tools (PROBAST) across four domains (population, risk factors, outcomes, and analysis) [32,33].”
MethodRévész, D., van Kuijk, S. M. J., Mols, F., van Duijnhoven, F. J. B., Winkels, R. M., Hoofs, H., Kant, I., Smits, L. J. M., Breukink, S. O., van de Poll-Franse, L. V., et al. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-1064-9 · Cited by 30
Prediction models in first-episode psychosis: systematic review and critical appraisal
Lee, R., Leighton, S., Thomas, L., Gkoutos, G. V., Wood, S. J., Fenton, S., Deligianni, F., Cavanagh, J., & Mallikarjun, P. K. (2022) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.219 · Cited by 32
Razaghizad, A., Oulousian, E., Randhawa, V. K., Ferreira, J. P., Brophy, J. M., Greene, S. J., Guida, J., Felker, G. M., Fudim, M., Tsoukas, M. A., et al. (2022) · Journal of the American Heart Association · doi:10.1161/jaha.121.024833 · Cited by 30
Zhang, H., Wang, Y., Wu, S., Ngo, J., Feng, Y., He, X., Zhang, Y., Wu, X., & Hong, D. (2022) · BMC Nephrology · doi:10.1186/s12882-022-03025-w · Cited by 30
Ramaekers, M., Viviers, C. G. A., Janssen, B. V., Hellström, T. A. E., Ewals, L. J. S., van der Wulp, K., Nederend, J., Jacobs, I., Pluyter, J. R., Mavroeidis, D., et al. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12134209 · Cited by 30
“To reduce such risks, the prediction model risk of bias assessment tool (PROBAST) criteria can be used to assess the risk of bias for prediction model studies [96].”
MethodHampton, J. S., Kenny, R. P. W., Rees, C. J., Hamilton, W. T., Eastaugh, C., Richmond, C., & Sharp, L. (2023) · EClinicalMedicine · doi:10.1016/j.eclinm.2023.102204 · Cited by 30
Artificial Intelligence and Child Abuse and Neglect: A Systematic Review
Lupariello, F., Sussetto, L., Trani, S. D., & Vella, G. D. (2023) · Children · doi:10.3390/children10101659 · Cited by 37
“This approach is a correct form of external validation (called temporal validation) because it tests the predictive AI algorithm on a sample collected in a later period [10].”
BackgroundMethodInfluentialClift, A. K., Tan, P. S., Patone, M., Liao, W., Coupland, C. A. C., Bashford-Rogers, R. J. M., Sivakumar, S., & Hippisley‐Cox, J. (2024) · British Journal of Cancer · doi:10.1038/s41416-024-02693-9 · Cited by 33
Prediction modelling studies for medical usage rates in mass gatherings: A systematic review
Remoortel, H. V., Scheers, H., Buck, E. D. J. D., Haenen, W., & Vandekerckhove, P. G. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234977 · Cited by 28
Silva, F. G., Costa, L. O. P., Hancock, M. J., Palomo, G. A., Costa, L. C., & da Silva, T. M. (2022) · Journal of physiotherapy · doi:10.1016/j.jphys.2022.03.009 · Cited by 32
Prediction Models for Bronchopulmonary Dysplasia in Preterm Infants: A Systematic Review
Peng, H., Zhan, Y.-L., Chen, Y., Jin, Z.-C., Liu, F., Wang, B., & Yu, Z. (2022) · Frontiers in Pediatrics · doi:10.3389/fped.2022.856159 · Cited by 29
A Systematic Review and Meta-Analysis of Prognostic Nomograms After UTUC Surgery
Pallauf, M., König, F., D’Andrea, D., Laukhtina, E., Mostafaei, H., Motlagh, R. S., Quhal, F., Aydh, A., Yanagisawa, T., Kawada, T., et al. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.907975 · Cited by 25
Risk prediction models for breast cancer: a systematic review
Zheng, Y., Li, J., Wu, Z. L., Li, H., Cao, M., Li, N., & He, J. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-055398 · Cited by 30
Clinical Risk Assessment and Prediction in Congenital Heart Disease Across the Lifespan
Opotowsky, A. R., Khairy, P., Diller, G., Kasparian, N. A., Brophy, J. M., Jenkins, K. J., Lopez, K. N., McCoy, A. B., Moons, P., Ollberding, N. J., et al. (2024) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2024.02.055 · Cited by 28
Cox, E. G. M., Wiersema, R., Eck, R. J., Kaufmann, T., Granholm, A., Vaara, S. T., Møller, M. H., van Bussel, B. C. T., Snieder, H., Pleijhuis, R. G., et al. (2022) · Critical Care Medicine · doi:10.1097/ccm.0000000000005712 · Cited by 32
“Prediction models are prognostic tools used to estimate the probability, based on clinical data, of a prespecified outcome, for example, mortality (1).”
MethodHuang, X., Chen, H., Meng, S., Pu, L., Xu, X., Xu, P., He, S., Hu, X., Li, Y., & Wang, G. (2024) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2024.104867 · Cited by 24
Fariña, R., Simonelli, A., Trombelli, L., Ettmayer, J. B., Schmid, J. L., & Ramseier, C. A. (2025) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.14156 · Cited by 33
Botelho, S. F., Pantuzza, L. L. N., Marinho, C. P., & Reis, A. M. M. (2020) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2020.08.002 · Cited by 26
Thurtle, D. R., Rossi, S. H., Berry, B., Pharoah, P. D. P., & Gnanapragasam, V. J. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029149 · Cited by 24
A systematic review of risk prediction models for perioperative mortality after thoracic surgery
Taylor, M., Hashmi, S. F., Martin, G. P., Shackcloth, M. J., Shah, R. D., Booton, R., & Grant, S. W. (2020) · Interactive Cardiovascular and Thoracic Surgery · doi:10.1093/icvts/ivaa273 · Cited by 24
Metadata
- Authors
- Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S.
- Affiliations
- Utrecht University (NL)
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- W1-W33
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907638671
- Cited by
- 1,669
- References
- 201
- Citations by year
- 2026: 359 · 2025: 330 · 2024: 253 · 2023: 236 · 2022: 193 · 2021: 175 · 2020: 96 · 2019: 24
- FWCI
- 56.12
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (publishedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- PROBAST, risk of bias assessment, applicability, prediction model, diagnostic model, prognostic model, model validation
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.