PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration
Open accessThe DOI matches its Crossref record; 194 of the 201 references it cites were found in a registry or an index.
Abstract
Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMeasuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Research works printed but not linked: 2
- 10.1177/0962280218785504.30032705 · doi:10.1177/0962280218785504.30032705
- Wong SS, 2003, AMIA Annu Symp Proc, 728
Cited by
Items 601–700 of 1,003
Wedig, I. J., Lennox, I. M., Petushek, E. J., McDaniel, J., Durocher, J. J., & Elmer, S. J. (2023) · European Journal of Applied Physiology · doi:10.1007/s00421-023-05352-8 · Cited by 11
“Accordingly, there have been several initiatives to improve prediction modeling research (Steyerberg and Vergouwe 2014; Moons et al. 2015, 2019).”
Siddique, S. M., Evans, C. V., Harhay, M. O., Johnson, E. S., Aysola, J., Weissman, G. E., Mull, N. K., Flores, E. J., Schmidt, H., Tipton, K. N., et al. (2023) · Health Equity · doi:10.1089/heq.2023.0035 · Cited by 10
“Additional guidance available in Moons et al. 26 Because model performance is quantified using a number of different measures, assessments of algorithmic bias may differ for each measure with respect to population characteristics (i.e., case mix) and varying condition prevalence or incidence.”
BackgroundMethodInfluentialZhou, B., Ai, W., & Cao, H. (2023) · International Journal of Orthopaedic and Trauma Nursing · doi:10.1016/j.ijotn.2023.101077 · Cited by 13
Quality assessment of radiomics models in carotid plaque: a systematic review
Hou, C., Li, S., Zheng, S., Liu, L., Nie, F., Zhang, W., & He, W. (2023) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims-23-712 · Cited by 11
“Blinded assessment of predictor and outcome is important not only in prognostic trails but also in diagnostic model studies to reduce the risk of bias (60 , 61) .”
Zahra, A., van Smeden, M., Abbink, E. J., van den Berg, J. M., Blom, M. T., van den Dries, C. J., Gussekloo, J., Wouters, F., Joling, K. J., Melis, R. J. F., et al. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111270 · Cited by 10
Self-citationLai, X., Liu, J., Zhang, T., Feng, L., Jiang, P., Kang, L., Liu, Q., & Gao, Y. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-039813 · Cited by 10
Wang, M., Li, N., Luo, Q., Wang, G., Heizhati, M., Wang, L., Wang, L., & Zhang, W. (2021) · Endocrine · doi:10.1007/s12020-021-02745-7 · Cited by 10
Geethadevi, G. M., Quinn, T., George, J., Anstey, K. J., Bell, J. S., & Cross, A. J. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014885 · Cited by 12
Risk prediction models for symptomatic patients with bladder and kidney cancer: a systematic review
Harrison, H., Usher‐Smith, J. A., Li, L., Roberts, L. E., Lin, Z., Thompson, R. E., Rossi, S. H., Stewart, G. D., Walter, F. M., Griffin, S. J., et al. (2021) · British Journal of General Practice · doi:10.3399/bjgp.2021.0319 · Cited by 11
Kailavasan, M., Berridge, C., Yuan, Y., Turner, A., Donaldson, J. F., & Biyani, C. S. (2022) · Journal of Pediatric Urology · doi:10.1016/j.jpurol.2022.05.013 · Cited by 10
Wang, M., Sajobi, T. T., Ismail, Z., Seitz, D. P., Chekouo, T., Forkert, N. D., Fischer, K., Mackie, A., Pearson, D. M., Patry, D. G., et al. (2022) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1002/trc2.12301 · Cited by 15
Liu, D., Guo, Z., Wang, J., Tian, E., Chen, J., Zhou, L., Kong, W., & Zhang, S. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11164745 · Cited by 10
Taheriyan, M., Nodehi, S. R. S., Kalhori, S. R. N., & Mohammadzadeh, N. (2022) · Expert Review of Hematology · doi:10.1080/17474086.2022.2042248 · Cited by 9
Niu, T., Zhou, X., Li, X., Liu, T., Liu, Q., Li, R., Liu, Y., & Dong, H. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.1060715 · Cited by 10
“The Prediction Model Risk of Bias Assessment Tool (PROBAST) score (39) is high for this article because it has no external validation.”
BackgroundGehringer, C. K., Martin, G. P., Hyrich, K. L., Verstappen, S. M. M., Sexton, J., Kristianslund, E. K., Provan, S. A., Kvien, T. K., & Sergeant, J. C. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.111239 · Cited by 11
Li, Y., Liu, Z., Liu, T., Li, J., Mei, Z., Fan, H., & Cao, C. (2024) · International Journal of Public Health · doi:10.3389/ijph.2024.1606913 · Cited by 12
Suliman, A., Masud, M. M., Serhani, M. A., Abdullahi, A. S., & Oulhaj, A. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-082654 · Cited by 10
Risk prediction algorithms for prolonged postoperative ileus: A systematic review
Liu, G. X. H., Milne, T. G. E., Xu, W., Varghese, C., Keane, C., O’Grady, G., Bissett, I. P., & Wells, C. I. (2024) · Colorectal Disease · doi:10.1111/codi.17010 · Cited by 14
“All included studies were assessed using the Prediction model Risk Of Bias ASsessment (PROBAST) Tool [ 23,24], a quality appraisal tool that identifies risk of bias (RoB) and applicability concerns in diagnostic and prognostic prediction model studies.”
MethodWu, C.-C., Poly, T. N., Weng, Y.-C., Lin, M., & Islam, M. M. (2024) · Diagnostics · doi:10.3390/diagnostics14151594 · Cited by 11
“The same two reviewers independently utilized the prediction model risk of bias assessment tool (PROBAST) to assess the methodological quality of each included ML model study [21].”
MethodMachine Learning for Predicting Stillbirth: A Systematic Review
Li, Q., Li, P., Chen, J., Ren, R., Ren, N., & Xia, Y. (2024) · Reproductive Sciences · doi:10.1007/s43032-024-01655-z · Cited by 10
“The two reviewers independently assessed risk of bias and the applicability of included studies using PROBAST, a tool that consists of 20 signaling questions in four domains: participants, predictors, outcomes, and analysis [18 ].”
Risk prediction model for in-stent restenosis following PCI: a systematic review
Xiang, Q., Xiong, X., Liu, S., Zhang, M., Li, Y., Wang, H., Wu, R., & Chen, L. (2024) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2024.1445076 · Cited by 14
In-Bed Monitoring: A Systematic Review of the Evaluation of In-Bed Movements Through Bed Sensors
Ocagli, H., Lanera, C., Borghini, C., Khan, N. M., Casamento, A., & Gregori, D. (2024) · Informatics · doi:10.3390/informatics11040076 · Cited by 14
Xia, Z., Cao, S., Li, T., Yuan, Q., & Zhong, Y. (2024) · Diabetes Metabolic Syndrome and Obesity · doi:10.2147/dmso.s489819 · Cited by 9
Risk prediction models for diabetic retinopathy: a systematic review
Huang, H., Wu, Y., Ye, H., Li, J., Chen, L., & Huang, X. (2025) · Frontiers in Endocrinology · doi:10.3389/fendo.2025.1556049 · Cited by 12
Large Language Models in Lung Cancer: Systematic Review
Zhong, R., Chen, S., Li, Z., Gao, T., Su, Y., Zhang, W., Liu, D., Gao, L., & Hu, K. (2025) · Journal of Medical Internet Research · doi:10.2196/74177 · Cited by 14
“PROBAST (Prediction Model Risk of Bias Assessment Tool) [18] is specifically designed to assess the risk of bias in studies involving predictive modeling and was applied accordingly.”
Tu, Y., Zhu, H., Zhang, X., Huang, S., & Tu, W. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-07401-2 · Cited by 16
“Each domain includes 2 to 9 signaling questions (totaling 20 questions), which guide the evaluation of bias risk [44].”
Data Analytics and Administrative Decision‐Making in Nursing Management: A Systematic Review
Darach, N., Kim, M. S., Wisesrith, W., & Collins, E. G. (2025) · Journal of Nursing Management · doi:10.1155/jonm/4344147 · Cited by 15
“…of Observational Studies in Epidemiology (STROBE) tool [28], guideline for good evaluation practice in health informatics (GEP-HI) [29], risk-of-bias tool for prediction models (PROBAST) [30], and the Quality Improvement Minimum Quality Criteria Set (QI-MQCS) [31] were applied for other studies.”
Li, Q., Chi, L., Zhao, W., Wu, L., Jiao, C., Zheng, X., Zhang, K., & Li, X. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1039794 · Cited by 12
“The quality of the literature was assessed using the Prediction model Risk Of Bias Assessment Tool (PROBAST).”
MethodInfluentialHou, F., Hou, Y., Sun, X., Lv, J., Jiang, H., Zhang, M., Liu, C., & Deng, Z.-Y. (2023) · PeerJ · doi:10.7717/peerj.15678 · Cited by 9
Lost, J., Verma, T., Jekel, L., von Reppert, M., Tillmanns, N., Merkaj, S., Petersen, G. C., Bahar, R. C., Gordem, A., Haider, M. A., et al. (2023) · American Journal of Neuroradiology · doi:10.3174/ajnr.a8000 · Cited by 13
Gerlee, P., Jöud, A., Sprečo, A., & Timpka, T. (2022) · Scientific Reports · doi:10.1038/s41598-022-16159-6 · Cited by 9
Zeng, G., Zhuang, J., Huang, H., Tian, M., Gao, Y., Liu, Y., & Yu, X. (2023) · Tsinghua Science & Technology · doi:10.26599/tst.2022.9010027 · Cited by 9
“Outcome prediction models may estimate event likelihood[2].”
Ashford, M. T., Neuhaus, J. M., Jin, C., Camacho, M. R., Fockler, J., Truran, D., Mackin, R. S., Rabinovici, G. D., Weiner, M. W., & Nosheny, R. L. (2020) · Alzheimer s & Dementia Diagnosis Assessment & Disease Monitoring · doi:10.1002/dad2.12102 · Cited by 8
Ye, L., Chen, Y., Xu, H., Wang, Z., Li, H., Qi, J., Wang, J., Yao, J., Liu, J., & Song, B. (2022) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2022.814388 · Cited by 8
Kaizu, Y., Tamura, S., Saito, H., Hayashi, S., Iwamoto, H., & Miyata, K. (2023) · The Journals of Gerontology Series A · doi:10.1093/gerona/glad205 · Cited by 9
Yan, C., Yang, Q., Li, R., Yang, A., Fu, Y., Wang, J., Li, Y., Cheng, Q., & Hu, S. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e20099 · Cited by 13
A systematic review of tools for predicting complications in patients with influenza-like illness
Marx, T., Khelifi, N., Xu, I., Ouellet, L., Poirier, A., Huard, B., Mallet, M., Bergeron, F., Boissinot, M., Bergeron, M. G., et al. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e23227 · Cited by 9
“Study quality was assessed using the prediction model risk of bias assessment tool (PROBAST) which rates study methodology and applicability to the review question as “ high ” , “ low ” or “ unclear ” risk of bias based on a set of questions and a scoring guide [10].”
MethodZeng, Q., Li, Z., Gui, S., Wu, J., Liu, C., Wang, T., Peng, D., & Zeng, G. (2024) · Journal of orthopaedic surgery · doi:10.1177/10225536241249591 · Cited by 11
“24 PROBAST assesses bias across four key domains: participants, predictors, outcomes, and analysis, assigning a rating of “ high, ” “ low, ” or “ unclear ” risk of bias to each domain.”
Risk Prediction Models for Gastric Cancer: A Scoping Review
Xu, L., Lyu, J., Zheng, X., & Wang, A. (2024) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s479699 · Cited by 9
Najdaghi, S., Davani, D. N., Shafie, D., & Alizadehasl, A. (2024) · International Urology and Nephrology · doi:10.1007/s11255-024-04257-5 · Cited by 8
Sankar, H., Alagarsamy, R., Lal, B., Rana, S. S., Roychoudhury, A., Barathi, A., & Ankush, A. (2024) · British Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.bjoms.2024.12.004 · Cited by 15
Yin, H., Wang, Y., Wang, H., Li, T., Xu, X., Li, F., & Huang, L. (2025) · Scientific Reports · doi:10.1038/s41598-025-85816-3 · Cited by 8
Jiang, C.-Q., Li, X., Zhou, Z., Xin, Q., & Lin, Y. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1532698 · Cited by 12
Lu, W., Wang, N., Fang, X., Yang, H., He, H., Qin, D., & Hua, F. (2025) · Journal of Dentistry · doi:10.1016/j.jdent.2025.105686 · Cited by 9
SDoH Impact on Periodontal Disease Using Machine Learning and Dental Records
Patel, J., Badi, M., Katiyar, R., Ogwo, C., Wiener, R. C., Tiwari, T., Sambamoorthi, U., & Folks, T. (2025) · Journal of Dental Research · doi:10.1177/00220345251328968 · Cited by 8
“We used the prediction model Risk of Bias Assessment Tool (PROBAST) to ensure minimal bias in results by evaluating the risk of bias and applicability of prediction model studies (Moons et al. 2019).”
Zhang, Z., Wang, S., Xu, Z., Sun, Y., Zhou, X., Zhou, R., Li, Q., & Wang, G. (2025) · Renal Failure · doi:10.1080/0886022x.2025.2500663 · Cited by 8
“…independently assessed the risk of bias and applicability of the literature using the Prediction model Risk Of Bias assessment tool (PROBaSt) [19]. the risk of bias assessment includes four domains: study population, predictors, outcomes, and statistical analysis. each domain is rated as…”
Risk prediction models for sarcopenia in elderly people: a systematic review and meta-analysis
Yin, J., Xu, Y., Cai, M., & Fang, X. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1589583 · Cited by 9
Pantelis, A. G., Epiphaniou, P., & Lapatsanis, D. P. (2025) · Artificial Intelligence Surgery · doi:10.20517/ais.2024.104 · Cited by 13
Evaluating YOLO for dental caries diagnosis: a systematic review and meta-analysis
Lam, Q. T., Minh, L. H. N., Lee, I., & Le, N. Q. K. (2025) · Evidence-Based Dentistry · doi:10.1038/s41432-025-01180-1 · Cited by 13
Risk prediction models for ovarian hyperstimulation syndrome: a systematic review and meta-analysis
Liu, J., Liu, F., Xu, W., Zhang, L., Tian, Q., Du, L., Li, S., & Zhang, M. (2025) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-025-07971-9 · Cited by 10
“As with PROBAST-2019 [13, 14], signalling questions are answered as “yes”, “probably yes”, “no”, “probably no”, “no information”, or, when appropriate, “not applicable”.”
InfluentialPredictive models for chemotherapy-induced oral mucositis: a systematic review
Tao, Y., Zeng, X. G., & Mao, H. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1608505 · Cited by 11
Selmer, N. W., Berthelsen, C. B., Grootven, B. V., Meyer, G., & Loft, M. I. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105188 · Cited by 14
Afsharnia, H., & Butt, J. (2025) · Journal of Manufacturing and Materials Processing · doi:10.3390/jmmp9100334 · Cited by 8
Deep learning models for ICU readmission prediction: a systematic review and meta-analysis
Koumantakis, E., Remoundou, K., Colombi, N., Fava, C., Roussaki, I. G., Visconti, A., & Berchialla, P. (2025) · Critical Care · doi:10.1186/s13054-025-05642-x · Cited by 11
“Two reviewers (EK and KR) independently assessed the risk of bias and applicability of the studies using the Prediction model Risk of Bias Assessment Tool (PROBAST, version 2019-05-15) [23].”
Ayele, A. D., Gedefaw, G. A., Alemu, B. K., Wang, Y., & wang, C. C. (2025) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2025-114065 · Cited by 13
“Two authors (ADA and GGA) independently evaluated study quality using the Prediction model Risk Of Bias ASsessment Tool (PROBAST), 39 which includes 20 signalling questions across four domains: participants, predictors, outcome and analysis.”
Xu, S., Wu, Y., Li, J., Pan, X., Zhang, X., Liu, Y., Zhang, F., & Tong, N. (2020) · Postgraduate Medicine · doi:10.1080/00325481.2020.1821234 · Cited by 8
Cai, S., Chen, Y., Zhao, S., He, D., Li, Y., Xiong, N., Li, Z., & Hu, S. (2022) · European Radiology · doi:10.1007/s00330-021-08533-1 · Cited by 8
Artificial intelligence in hepatFIGopancreaticobiliary surgery: a systematic review
Bektaş, M., Zonderhuis, B. M., Marquering, H. A., Pereira, J. C., Burchell, G. L., & van der Peet, D. L. (2022) · Artificial Intelligence Surgery · doi:10.20517/ais.2022.20 · Cited by 10
Khuong, J. N., Liu, Z., Campbell, R. J., Jackson, S. M., Caruana, C. B., Ramson, D. M., Penny‐Dimri, J. C., & Perry, L. A. (2022) · General Thoracic and Cardiovascular Surgery · doi:10.1007/s11748-022-01888-2 · Cited by 8
“Two reviewers (J.N.K. and L.A.P.) independently assessed the risk of bias of the included studies using the standardised Prediction model study Risk of Bias Assessment Tool (PROBAST) [12].”
MethodChifor, R., Arsenescu, T., Dascalu, L. M., & Badea, A. F. (2022) · Romanian Journal of Stomatology · doi:10.37897/rjs.2022.3.7 · Cited by 9
Chen, R., Zheng, J., Li, C., Chen, Q., Zeng, Z., Li, L., Chen, M., & Zhang, S. (2023) · Frontiers in Immunology · doi:10.3389/fimmu.2023.1215116 · Cited by 10
Daines, L., Bonnett, L. J., Tibble, H., Boyd, A., Thomas, R., Price, D. B., Turner, S., Lewis, S. C., Sheikh, A., & Pinnock, H. (2023) · Wellcome Open Research · doi:10.12688/wellcomeopenres.19078.1 · Cited by 9
Chen, Y.-L., Nguyen, P., Chien, C., Hsu, M., Liou, D.-M., & Yang, H. (2023) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2023.111033 · Cited by 13
Su, Z., Yu, X., He, Y., Sha, T., Guo, H., Tao, Y., Liao, L., Zhang, Y., Lu, G., Lu, G., et al. (2024) · BMC Pulmonary Medicine · doi:10.1186/s12890-024-03374-2 · Cited by 10
“Moreover, the models’ applicability to the intended population and setting were assessed by the Prediction Model Risk of Bias Assessment Tool (PRO-BAST) (Appendix 3) [32, 33].”
MethodModels for predicting risk of endometrial cancer: a systematic review
Forder, B. H., Ardasheva, A., Atha, K., Nentwich, H.-J., Abhari, R. E., & Kartsonaki, C. (2025) · Diagnostic and Prognostic Research · doi:10.1186/s41512-024-00178-0 · Cited by 8
Development of a novel risk score for the prediction of critical illness amongst COVID‐19 patients
Bellos, I., Lourida, P., Argyraki, A., Korompoki, E., Zirou, C., Kokkinaki, I., & Pefanis, A. V. (2020) · International Journal of Clinical Practice · doi:10.1111/ijcp.13915 · Cited by 8
Restrepo‐Escobar, M., Granda, P. A., Aguirre–Acevedo, D. C., Zapata, J. H., Vásquez, G. M., & Jaimes, F. A. (2021) · Lupus · doi:10.1177/0961203320983462 · Cited by 7
Ling, J. E. H., Coughlan, T., Polkinghorne, K. R., & Kanellis, J. (2021) · BMC Gastroenterology · doi:10.1186/s12876-021-01655-2 · Cited by 9
Takada, T., Hoogland, J., van Lieshout, C., Schuit, E., Collins, G. S., Moons, K. G. M., & Reitsma, J. B. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.11.033 · Cited by 8
Self-citationAdeoye, J., Alkandari, A. S., Tan, J., Wang, W., Zhu, W., Thomson, P. J., Zheng, L. W., Choi, S. W., & Su, Y. (2022) · Journal of Oral Pathology and Medicine · doi:10.1111/jop.13293 · Cited by 8
Predictive models for the incidence of Parkinson’s disease: systematic review and critical appraisal
Chen, Y., Gao, Y., Sun, X., Liu, Z., Zhang, Z., Qin, L., Song, J., Wang, H., & Wu, X. (2022) · Reviews in the Neurosciences · doi:10.1515/revneuro-2022-0012 · Cited by 10
Real-World Practice of Gastric Cancer Prevention and Screening Calls for Practical Prediction Models
He, S., Sun, D., Li, H., Cao, M., Yu, X., Lei, L., Peng, J., Li, J., Li, N., & Chen, W. (2022) · Clinical and Translational Gastroenterology · doi:10.14309/ctg.0000000000000546 · Cited by 11
“To assess the quality of the included studies, we used the Prediction model Risk of Bias Assessment Tool (PROBAST) to evaluate the ROB and applicability of each prediction model through signaling questions in 4 domains of participants, predictors, outcome, and statistical analysis (applicability assessment focuses on…”
MethodSufriyana, H., Wu, Y., & Su, E. C. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.02.020 · Cited by 12
Tanaka, M. D., Geubels, B. M., Grotenhuis, B. A., Marijnen, C. A. M., Peters, F. P., van der Mierden, S., Maas, M., & Couwenberg, A. M. (2023) · Cancers · doi:10.3390/cancers15153945 · Cited by 8
“PROBAST was used for risk of bias (ROB) assessment and evaluation of concerns regarding the applicability of a model [42,43].”
MethodWilson, J., Chowdhury, F., Hassan, S., Harriss, E., Alves, F. P., Dahal, P., Stepniewska, K., & Guerin, P. J. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-075597 · Cited by 9
“In accordance with expert guidance on the methodology of prediction model research, 14 21 22 we define a prognostic model as a multivariable model (including two or more predictors) where the intention is to predict outcomes at the individual patient level.”
MethodLessons learned from developing a COVID‐19 algorithm governance framework in Aotearoa New Zealand
Wilson, D., Tweedie, F., Rumball‐Smith, J., Ross, K., Kazemi, A., Galvin, V., Dobbie, G., Dare, T., Brown, P., & Blakey, J. (2022) · Journal of the Royal Society of New Zealand · doi:10.1080/03036758.2022.2121290 · Cited by 12
Mjåset, C., Solberg, T. K., Zwart, J., Småstuen, M. C., Kolstad, F., & Grotle, M. (2022) · Acta Neurochirurgica · doi:10.1007/s00701-022-05440-2 · Cited by 12
Chen, J., Lu, G., Wang, Z., Zhang, J., Ding, J., Zeng, Q., Chai, L., Zhao, L., Yu, H., & Li, Y. (2023) · The Laryngoscope · doi:10.1002/lary.30931 · Cited by 11
“27,28 Organized into 20 signaling questions across 4 domains, including participants, predictors, outcomes, and analysis, each domain is scored and given a biased assessment of low, high, or unclear, with a study being deemed high risk of bias if any domain is determined to have a high score.”
BackgroundHua, Y., Yan, C., Zhou, C., Zheng, Q., Li, D., & Tu, P. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2023.1224658 · Cited by 7
Lifestyle predictors of colorectal cancer in European populations: a systematic review
Mertens, E., Keuchkarian, M., Vasquez, M. S., Vandevijvere, S., & Peñalvo, J. L. (2024) · BMJ Nutrition Prevention & Health · doi:10.1136/bmjnph-2022-000554 · Cited by 10
“Twitter José L Peñalvo @JosePenalvo”
Ahmad, M., Tan, M. K. H., Bergman, H., Shalhoub, J., & Davies, A. H. (2024) · JVS-Vascular Insights · doi:10.1016/j.jvsvi.2024.100057 · Cited by 8
Giraldo‐Roldán, D., Araújo, A. L. D., Moraes, M. C., da Silva, V. M., Ribeiro, E. C. C., Cerqueira, M., Saldivia‐Siracusa, C., Sousa‐Neto, S. S., Pérez‐de‐Oliveira, M. E., Lopes, M. A., et al. (2024) · Journal of Oral Pathology and Medicine · doi:10.1111/jop.13548 · Cited by 9
Common Critiques and Recommendations for Studies in Neurology Using Machine Learning Methods
Speiser, J. L., Kerr, W. T., & Ziegler, A. (2024) · Neurology · doi:10.1212/wnl.0000000000209861 · Cited by 11
“27 Twelve articles used logistic regression for model development, and 1 article used ML 7 | October 8, 2024”
Aramruang, T., Malhotra, A., Numthavaj, P., Looareesuwan, P., Anothaisintawee, T., Dejthevaporn, C., Sirirutbunkajorn, N., Attia, J. R., & Thakkinstian, A. (2024) · The Journal of Headache and Pain · doi:10.1186/s10194-024-01874-4 · Cited by 9
“According to PROBAST, this predictor is unavailable at the point of model application and integral to outcome determination, resulting in a high ROB and concerns about applicability in clinical settings [27, 28].”
MethodInfluentialMethodological quality assessment tools for diagnosis and prognosis research: overview and guidance
Kaul, T., Kellerhuis, B. E., Damen, J. A., Schuit, E., Jenniskens, K., van Smeden, M., Reitsma, J. B., Hooft, L., Moons, K. G. M., & Yang, B. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111609 · Cited by 9
Self-citationPrediction of Preeclampsia Using Machine Learning: A Systematic Review
Malik, V., Agrawal, N., Prasad, S., Talwar, S., Khatuja, R., Jain, S., Sehgal, N. P., Malik, N., Khatuja, J., & Madan, N. (2024) · Cureus · doi:10.7759/cureus.76095 · Cited by 10
Gao, Y., Yuan, B., Fan, P., Li, M., & Chen, J. (2024) · Medicine · doi:10.1097/md.0000000000040588 · Cited by 7
Xu, J., Zhao, X., Li, F., Xiao, Y., & Li, K. (2024) · Journal of Clinical Nursing · doi:10.1111/jocn.17577 · Cited by 10
Zhang, Y., Yan, C., Lu, G., Diao, H., Liu, X., Ma, Q., Yu, H., Yang, L., & Li, Y. (2025) · Neurosurgical Review · doi:10.1007/s10143-025-03346-y · Cited by 7
Larsudd-Kåverud, J., Åkervall, S., Molin, M., Nilsson, I. E. K., Steyerberg, E. W., Milsom, I., & Gyhagen, M. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111782 · Cited by 9
Ram, M., Afrash, M. R., Moulaei, K., Esmaeeli, E., Khorashadizadeh, M. S., Garavand, A., Amiri, P., & Sabahi, A. (2025) · Technology in Cancer Research & Treatment · doi:10.1177/15330338251341053 · Cited by 8
Chen, X., Huang, Y., Jessney, B., Sangha, J., Gu, S. Z., Schönlieb, C., Bennett, M. R., & Roberts, M. T. (2025) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztaf053 · Cited by 9
Sánchez, J., Caraballo, A., Chérrez-Ojeda, I., García, E., Larco, J. I., Ramon, G. D., Velásquez‐Lopera, M. M., & Jaimes, F. A. (2025) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2025.05.051 · Cited by 8
Ma, Y., Li, M., & Wu, H. (2025) · Journal of Medical Internet Research · doi:10.2196/68872 · Cited by 7
“We used the Prediction Model Risk of Bias Assessment Tool (PROBAST) to assess the risk of bias and applicability concerns of each included study, using a total of 20 signaling questions in four domains—participants, predictors, outcome, and analysis [22].”
Diagnosis models to predict peripheral arterial disease: a systematic review and meta analysis
Quan, X., Xiong, H., Liu, X., Song, P., Wang, D., Chen, Q., Hu, X., & Shi, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-10459-3 · Cited by 9
“Future modeling studies should comprehensively enhance methodological quality by adhering to PROBAST guidelines for designing, constructing, evaluating, and validating models.”
InfluentialKnauer, R., Koddenbrock, M., Wallsberger, R., Brisson, N. M., Duda, G. N., Falla, D., Evans, D. W., & Rodner, E. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736818 · Cited by 19
“Listing 1 shows our prompting template for building a zero-shot decision tree T .”
InfluentialHmido, S. B., Rahim, H. A., Ingwersen, E. W., Burchell, G. L., Hompes, R., van der Peet, D. L., Sonneveld, E., Geert Kazemier, M., & Daams, F. D. (2025) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2025.110367 · Cited by 10
AI radiomics predicts spatial glioma recurrence on preoperative MRI: a systematic review
Bijlsma, S., Maal, T., Rubbert, C., Mannil, M., Meijer, F. J. A., van der Kolk, A. G., de Jong, G., & Henssen, D. J. H. A. (2025) · European Journal of Radiology · doi:10.1016/j.ejrad.2025.112412 · Cited by 10
Zhang, L., Cao, X., Yang, Y., Fu, S., Jia, Y., Hu, W., & Xiang, F. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1629369 · Cited by 7
Jian, R., Zhang, J., Zeng, Y., Zhou, T., Wu, Y., Wu, L., Yu, Y., & Xi, C. (2025) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2025.1659184 · Cited by 7
Metadata
- Authors
- Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S.
- Affiliations
- Utrecht University (NL)
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- W1-W33
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907638671
- Cited by
- 1,669
- References
- 201
- Citations by year
- 2026: 359 · 2025: 330 · 2024: 253 · 2023: 236 · 2022: 193 · 2021: 175 · 2020: 96 · 2019: 24
- FWCI
- 56.12
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (publishedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- PROBAST, risk of bias assessment, applicability, prediction model, diagnostic model, prognostic model, model validation
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.