PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration
Open accessThe DOI matches its Crossref record; 194 of the 201 references it cites were found in a registry or an index.
Abstract
Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMeasuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Research works printed but not linked: 2
- 10.1177/0962280218785504.30032705 · doi:10.1177/0962280218785504.30032705
- Wong SS, 2003, AMIA Annu Symp Proc, 728
Cited by
Items 801–900 of 1,003
Fallahpour, B., Dastjerdi, G., Akbarian, E., Emarati, A., Sadr, Z., Dastgheib, S. A., Shahbazi, A., Bahrami, R., Golshan-Tafti, M., Shiri, A., et al. (2026) · Egyptian Pediatric Association Gazette · doi:10.1186/s43054-026-00530-7 · Cited by 5
Pennisi, F., Pinto, A., Cozzolino, C., Cozza, A., Rezza, G., Signorelli, C., Baldo, V., & Gianfredi, V. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8040093 · Cited by 5
Genecand, L., Boucly, A., Beurnier, A., Jaïs, X., Degano, B., Prevot, G., Gerges, C., Lamblin, N., Bauer, F., Savale, L., et al. (2026) · The Journal of Heart and Lung Transplantation · doi:10.1016/j.healun.2026.05.028 · Cited by 5
Evidence-Based Utilization of Prognostic Prediction Models in Cardiovascular Medicine
Iwakami, N., Nagai, T., Furukawa, T. A., Nishimura, K., & Anzai, T. (2019) · Circulation Reports · doi:10.1253/circrep.cr-19-0111 · Cited by 6
Hsu, Y., & Kao, Y. (2022) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000952 · Cited by 9
Liu, Z., Jia, N., Zhang, Q., & Liu, W. (2023) · Journal of Diabetes & Metabolic Disorders · doi:10.1007/s40200-023-01330-1 · Cited by 5
“PROBAST is a bias risk assessment tool developed by Moons Karel in 2019, specifically designed for evaluating the development, validation, or updating of predictive models [18].”
Cheng, P., Yang, P., Zhang, H., & Wang, H. (2023) · Emergency Medicine International · doi:10.1155/2023/6780941 · Cited by 9
“Te two researchers used the Prediction Model Risk of Bias ASsess-ment Tool (PROBAST) [13], which was created expressly to evaluate diagnostic and prognostic prediction models, to evaluate the risk of bias and application of the literature that had been gathered.”
MethodZhou, L., Wang, L., Liu, G., & Cai, E. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02544-x · Cited by 6
“…will be conducted by employing the CHARMS checklist (critical appraisal and data extraction for systematic reviews of prediction modeling studies) [31] and the PROBAST (Prediction model Risk Of Bias ASsessment Tool) with four domains (i.e., participants, predictors, outcome, and analysis) [32, 33].”
Methodvan Eijk, J., Luijken, K., Trappenburg, J. C. A., Jaarsma, T., & Asselbergs, F. W. (2024) · Netherlands Heart Journal · doi:10.1007/s12471-024-01886-4 · Cited by 6
“To implement this, studies should be set up to derive and externally validate a prognostic model that predicts risk of (re)hospitalisation in an HF population at the time of telemedicine initiation [33, 34].”
MethodPrediction models for cognitive frailty in community-dwelling older adults: A scoping review
Zhang, Y., Xue, H., Xia, H., & Jiang, X. (2024) · Geriatric Nursing · doi:10.1016/j.gerinurse.2024.09.019 · Cited by 5
Uthman, A. T., Abouelenen, H., Khan, S., Bseiso, O., & Al-Rawi, N. H. (2025) · Imaging Science in Dentistry · doi:10.5624/isd.20240139 · Cited by 5
“The Prediction Model Risk of Bias Assessment Tool (PROBAST) for nonrandomized studies 10 was utilized to evaluate the risk of bias and applicability of the studies (Table 1).”
Tian, C. H., Liu, L., Huang, Y. F., Yang, H. J., Lai, Y. Y., Li, C. L., Gan, D., & Yang, J. (2025) · Human Reproduction · doi:10.1093/humrep/deaf013 · Cited by 7
Ren, M., Guo, H., Guo, Y., Guo, W., & Zhu, L. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-05961-2 · Cited by 5
“Two researchers (M.R. and W.G.) utilized the PROBAST [14] evaluation tool to assess both the ROB and the applicability of studies independently, and if there was disagreement, a third researcher (L.Z.) was consulted to reach a consensus.”
InfluentialLi, F., Zhang, J.-J., Ruan, G.-J., Zhan, Y.-Y., Qiu, A.-H., Ye, M., Xiang, Q., & Fang, H. (2025) · International Journal of Cardiology · doi:10.1016/j.ijcard.2025.133603 · Cited by 5
Xiao, W., Gu, D., Zhang, M., Liao, J., Xu, T., Deng, T., & Zhao, Y. (2025) · Journal of Clinical Nursing · doi:10.1111/jocn.17855 · Cited by 7
Han, X., Shan, L., Xu, R., Zhou, J., & Lu, M. (2025) · Academic Radiology · doi:10.1016/j.acra.2025.06.030 · Cited by 6
Takada, T., Damen, J. A., Tambaş, M., Spijker, R., Steenbakkers, R. J. H. M., Sharabiani, M., Clementel, E., Langendijk, J. A., Moons, K. G. M., & Schuit, E. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014745 · Cited by 5
Self-citationGao, T., Chen, F., & Li, M. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.899153 · Cited by 5
Development of a Prediction Model for Significant Adverse Outcome After Spine Surgery
Coia, M., & Baker, J. F. (2022) · Global Spine Journal · doi:10.1177/21925682221110819 · Cited by 5
Wang, Y., Xiao, Y., Yang, Q., Wang, F., Wang, Y., & Yuan, C. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-064566 · Cited by 7
Prognostic models for COVID-19 needed updating to warrant transportability over time and space
van Klaveren, D., Zanos, T. P., Nelson, J., Levy, T. J., Park, J. G., Retel Helmrich, I. R. A., Rietjens, J. A. C., Basile, M. J., Hajizadeh, N., Lingsma, H. F., et al. (2022) · BMC Medicine · doi:10.1186/s12916-022-02651-3 · Cited by 5
“All reviewed models demonstrated a high or unclear risk of bias when evaluated using the prediction model risk of bias assessment tool (PROBAST) [22].”
MethodInfluentialGu, Y., Wu, W., Kong, C., Luo, Q.-Q., Li, R., Tan, X., & Zhang, Q. (2022) · The Journals of Gerontology Series A · doi:10.1093/gerona/glac256 · Cited by 5
“By including a limited number of valuable predictors, our study provided a simple and practical model (29) for mortality estimation.”
InfluentialDiniz, J. M., Vasconcelos, H., Souza, J., Rb-Silva, R., Ameijeiras-Rodriguez, C., & Freitas, A. (2023) · JMIR Research Protocols · doi:10.2196/45823 · Cited by 4
“For each selected paper, the CHARMS [88] and PROBAST (Prediction Model Risk of Bias Assessment Tool) [89] checklists will be used to assess the risk of bias of included research works.”
Sheikh, Z., Potter, E., Li, Y., Drysdale, S. B., Wildenbeest, J., Robinson, H., McGinley, J., Lin, G.-L., Öner, D., Aerssens, J., et al. (2023) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiad388 · Cited by 5
Lu, Z., Zhou, X., Fu, L., Li, Y., Tian, T., Liu, Q., & Zou, H. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073375 · Cited by 6
Talimtzi, P., Ntolkeras, A., Kostopoulos, G., Bougioukas, K. I., Pagkalidou, E., Ouranidis, A., Pataka, A., & Haidich, A. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111264 · Cited by 5
Yang, H., Zhang, Y., Heng, F., Li, W., Feng, Y., Tao, J., Wang, L., Zhang, Z., Li, X., & Lu, Y. (2024) · Asian Nursing Research · doi:10.1016/j.anr.2024.04.012 · Cited by 6
Zhu, M., Han, F., Gao, J., Yang, J., Yin, L.-L., Du, Z., & Zhang, J. (2024) · Academic Radiology · doi:10.1016/j.acra.2024.06.020 · Cited by 7
Ön, B. I., Havla, J. B., & Mansmann, U. (2024) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2024.106247 · Cited by 5
Luo, A., Chen, W., Zhu, H., Xie, W., Chen, X., Liu, Z., & Xin, Z. (2024) · Journal of Medical Internet Research · doi:10.2196/60888 · Cited by 7
“We selected the widely recognized Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) [29] and Prediction model Risk Of Bias Assessment Tool (PROBAST) [30] as methodological quality assessment tools.”
Blekić, W., D’Hondt, F., Shalev, A. Y., & Schultebraucks, K. (2025) · Nature Mental Health · doi:10.1038/s44220-024-00365-4 · Cited by 11
“These domains help in identifying methodological flaws and provide guidance on assessing the applicability of the model 91 .”
InfluentialChen, S.-H., Wu, C.-C., & Chen, K.-F. (2025) · Biomedical Journal · doi:10.1016/j.bj.2025.100853 · Cited by 5
“To ensure the reliability and validity, we applied the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) [27] tool for quality assessment of diagnostic accuracy studies and the Prediction model Risk Of Bias Assessment Tool (PROBAST) [28] tool for quality and bias assessment.”
Beneyto‐Ripoll, C., Palazón‐Bru, A., Llópez‐Espinós, P., Martínez‐Díaz, A. M., Gil-Guillén, V. F., & Carbonell‐Torregrosa, M. Á. (2021) · International Journal of Clinical Practice · doi:10.1111/ijcp.14044 · Cited by 5
Moriarty, A. S., Paton, L. W., Snell, K. I. E., Riley, R. D., Buckman, J. E. J., Gilbody, S. M., Chew-Graham, C. A., Ali, S., Pilling, S., Meader, N., et al. (2021) · Diagnostic and Prognostic Research · doi:10.1186/s41512-021-00101-x · Cited by 5
Self-citationPrognostic model for predicting survival in very preterm infants: an external validation study
van Beek, P. E., Groenendaal, F., Onland, W., Koole, S., Dijk, P. H., Dijkman, K. P., van den Dungen, F., van Heijst, A., Kornelisse, R. F., Schuerman, F., et al. (2021) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17010 · Cited by 6
Clinical informatics solutions in COVID-19 pandemic: Scoping literature review
Ganjali, R., Eslami, S., Samimi, T., Sargolzaei, M., Firouraghi, N., MohammadEbrahimi, S., Khoshrounejad, F., & Kheirdoust, A. (2022) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2022.100929 · Cited by 8
Wang, M., Heizhati, M., Li, N., Yao, X., Luo, Q., Lin, M., Hong, J., Ma, Y., Wang, R., Sun, L., et al. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.777946 · Cited by 6
Systematic Reviews of Prediction Models
Collins, G. S., Moons, K. G. M., Debray, T. P. A., Altman, D. G., & Riley, R. D. (2022) · Wiley eBooks · doi:10.1002/9781119099369.ch18 · Cited by 8
Self-citationIsmail, S. R., Khalil, M. K. N., Mohamad, M. S. F., & Shah, S. A. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.921044 · Cited by 10
Clinical predictive models in equine medicine: A systematic review
Cummings, C. O., Krucik, D. D. R., & Price, E. (2022) · Equine Veterinary Journal · doi:10.1111/evj.13880 · Cited by 7
De, A. (2023) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2023.2169752 · Cited by 6
“If the Artificial Intelligence medical test uses the clinical reference standard as part of the input for the software in order to derive the output provided to the end user, then this could introduce bias for the statistical results (U.S. FDA 2007; Moons et al. 2019).”
BackgroundMethodChavarro‐Carvajal, D. A., Sanchez, D. C., Vargas-Beltrán, M. P., Venegas‐Sanabria, L. C., & Múñoz, Ó. (2023) · Colombia medica · doi:10.25100/cm.v54i1.5304 · Cited by 7
“574 Modified Lawton, Median (IQR) 7 (4-11) 6 (3-10) 0.”
A systematic review of risk prediction model of venous thromboembolism for patients with lung cancer
Wang, Y., Li, Q., Zhou, Y., Dong, Y., Li, J., & Liang, T. (2024) · Thoracic Cancer · doi:10.1111/1759-7714.15219 · Cited by 5
“27,28”
BackgroundOn the aggregation of published prognostic scores for causal inference in observational studies
Nguyen, T., Collins, G. S., Pellegrini, F., Moons, K. G. M., & Debray, T. P. A. (2020) · Statistics in Medicine · doi:10.1002/sim.8489 · Cited by 5
Self-citationHuman and machine learning pipelines for responsible clinical prediction using high-dimensional dataPreprint
Sufriyana, H., Wu, Y., & Su, E. C. (2021) · Research Square · doi:10.21203/rs.3.pex-1655/v1 · Cited by 5
Harnessing Moravec’s paradox in health care: a new era of collaborative intelligence
Loaiza-Bonilla, A., & Penberthy, S. (2025) · NEJM AI · doi:10.1056/aip2500005 · Cited by 4
Hou, Y., Feng, S., Tong, B., Lu, S., & Wang, H. (2025) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.18119 · Cited by 4
“Overall, most models had a high risk of bias, but most were rated as low concern, suggesting that methodological quality issues included in the models limited their performance and application [29].”
Sun, D., Wen, Y., Yu, Q., Long, Y., Liu, Y., Zhou, Y. X., & Yu, Y. (2025) · European Spine Journal · doi:10.1007/s00586-025-08785-1 · Cited by 7
“The risk of bias and applicability of the include studies were assessed using the PROBAST [17, 18].”
Prediction models for intraventricular hemorrhage in very preterm infants: a systematic review
Xiong, P., Wei, Y., Li, L., Kang, H., Yu, Z., Tang, H., & Pu, Y. (2025) · Frontiers in Pediatrics · doi:10.3389/fped.2025.1605145 · Cited by 6
Chen, Y., Hu, Y.-X., Wang, J., Sun, J., Hu, B., Huang, K., He, Z., Liang, C., & Lin, Y. (2025) · Annals of Medicine · doi:10.1080/07853890.2025.2519673 · Cited by 4
“Calibration curves were used to evaluate the calibration ability [19].”
Liang, W., Yang, C., Qin, L.-H., Lan, M.-N., & Liao, P. (2025) · Academic Radiology · doi:10.1016/j.acra.2025.07.008 · Cited by 4
Zhou, X., Wu, B., Tang, D., He, Y., & Fang, L. (2025) · Digestive Diseases and Sciences · doi:10.1007/s10620-025-09217-2 · Cited by 6
“Two researchers trained in evidence-based medicine assessed the risk of bias and applicability of the included studies using the Prediction Model Risk Of Bias Assessment Tool (PROBAST), as recommended by Cochrane [11].”
Kamarova, S., Davidson, S. R. E., Williams, C. M., Leite, M. N., & Kamper, S. J. (2025) · Trauma Violence & Abuse · doi:10.1177/15248380251358224 · Cited by 7
“All eligible studies were independently assessed by two authors for Risk of Bias (RoB) using the Prediction model Risk of Bias Assessment Tool (PROBAST) (Wolff et al., 2019).”
InfluentialLin, Z., Wang, H., Ma, C., Ju, R., Cao, Y., & Lin, P. (2025) · Frontiers in Neurology · doi:10.3389/fneur.2025.1632682 · Cited by 4
Shi, N., Li, J., Shang, M., Zhang, W. L., Xu, K., Li, Y., & Liang, L. (2025) · Journal of Medical Internet Research · doi:10.2196/81328 · Cited by 4
Wei, X., Norsuddin, N. M., Hamid, H. A., Azmi, M. I., Zhang, G., & Tian, J. (2025) · Health Science Reports · doi:10.1002/hsr2.71350 · Cited by 6
van de Mortel, L. A., & van Wingen, G. A. (2025) · Molecular Psychiatry · doi:10.1038/s41380-025-03336-y · Cited by 10
“When performing such analyses, quality assessment tools such as the REFORMS and PROBAST-AI questionnaires should be used to assess methodological biases in selected studies [8, 9].”
Ivanovic, V., Sujan, M. A. J., Mengshoel, O. J., & Moholdt, T. T. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106158 · Cited by 11
Langenhuijsen, L. F. S., Derksen, D. C. L., Milders, J., van der Horst, S. F. B., van Diepen, M., Trines, S. A. I. P., den Exter, P. L., Klok, F. A., Rotmans, J. I., & de Jong, Y. (2025) · Kidney Medicine · doi:10.1016/j.xkme.2025.101200 · Cited by 5
“27 Conflicts in scoring were discussed until consensus was reached.”
Josephson, C. B., Beniczky, S., Denaxas, S., Ikeda, A., Jehi, L. E., Mwesige, A. K., Jetté, N., Jones, G. D., Ryvlin, P., Sen, A., et al. (2025) · Epilepsia · doi:10.1002/epi.70058 · Cited by 8
Monteiro, S. M., Bota, P. J., Cunha, P. S., Junior, M. N. S. D. O., Laranjo, S. M., & Silva, H. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2026.109249 · Cited by 5
Shanmugaraj, A., Khalid, B., Kumar, M. V., Kunze, K. N., & Sheth, U. (2026) · Shoulder & Elbow · doi:10.1177/17585732251412368 · Cited by 4
“20,21 PROBAST assesses four domains via 20 signaling questions: participants, predictors, outcomes, and analysis.”
Prediction models for longitudinal trajectories of depression and anxiety: a systematic review
Fairweather, S., Fraser, H., Lam, N., Gilbody, S., Paton, L. W., Jones, H. J., & Khandaker, G. M. (2026) · Journal of Affective Disorders · doi:10.1016/j.jad.2026.121255 · Cited by 7
“Detailed guidance about how to make judgements for each signalling question can be found in the paper describing the original PROBAST tool (Moons et al., 2019).”
Liang, Y., Dai, A., Luo, M., Zheng, Z., Shen, J., Su, Y., & Li, Z. (2026) · Journal of Medical Internet Research · doi:10.2196/79729 · Cited by 5
“PROBAST consisted of four domains: participants, predictors, outcomes, and analysis [26].”
Partheniadis, I., Talimtzi, P., Nikolakopoulou, A., & Haidich, A. (2026) · Diagnostic and Prognostic Research · doi:10.1186/s41512-026-00218-x · Cited by 7
“Insufficient transparency not only hampers the ability to assess how a study was conducted or interpret its findings but also limits opportunities for independent validation by other researchers [7, 8].”
Large Language Models in Colorectal Cancer Care and Clinical Decision Support: Systematic Review
Tian, J., Lou, Q., Wang, X., xuhangying, Mei, H., & Yu, Y. (2026) · Journal of Medical Internet Research · doi:10.2196/89862 · Cited by 5
Rodriguez, R. W., Joseph, H., Macrito, R., Lee, T. A., & Sweiss, K. I. (2023) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxad147 · Cited by 7
Kokkinakis, S., Kritsotakis, E. I., Paterakis, K. N., Karali, G., Malikides, V., Kyprianou, A., Papalexandraki, M., Anastasiadis, C. S., Zoras, O., Drakos, N., et al. (2023) · European Journal of Trauma and Emergency Surgery · doi:10.1007/s00068-023-02351-4 · Cited by 5
“As recommended, we imputed the missing values before model development using Multiple Imputation by Chained Equations [11, 18].”
MethodInfluentialTanga, C., López‐Lázaro, S., Soto-Álvarez, C., & Viciano, J. (2024) · Forensic Science International · doi:10.1016/j.forsciint.2024.112143 · Cited by 7
Prediction models for skin tears in the elderly: A systematic review and meta-analysis
Fan, S., Jiang, H., Shen, J., Lin, H., Yang, L., Yu, D., Zhang, M., Zheng, N., & Chen, L. (2024) · Geriatric Nursing · doi:10.1016/j.gerinurse.2024.06.030 · Cited by 4
Miller, R., Jackson, L. H., Vilić, D., Boyce, L., & Shuaib, H. (2025) · Journal of Hand Surgery (European Volume) · doi:10.1177/17531934241312896 · Cited by 4
“…a data scientist for adherence to the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) (Mongan, Moy, & Kahn, 2020), and the Prediction Model Study Risk of Bias Assessment Tool checklist was used to assess studies for bias and applicability (Moons et al., 2019; Wolff et al., 2019).”
Pu, Y., Yang, J., Shui, L., Tang, Q., Zhang, X., & Liu, G. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1502404 · Cited by 7
“PROBAST, as a tool speci fi cally designed to assess the risk of bias in predictive model studies, performs a meticulous assessment from four key dimensions: participants, predictors, outcome, analysis, each containing a series of questions to determine the possible risk of bias in studies.”
InfluentialKelebie, M. A., Fentahun, S., Tadesse, G., Nakie, G., Takelle, G. M., Fasil, B., Rtbey, G., Muche, M., Gobezie, M., Alazar, A., et al. (2025) · BMC Public Health · doi:10.1186/s12889-025-22095-7 · Cited by 6
“Despite legitimate concerns about the pooled prevalence of long-term outcomes and the well-appreci-ated heterogeneity in the estimates, we identified pooled prevalence for long-term outcomes [74–76].”
Li, J., Li, G., Liu, Z., Wang, J., Jia, X., Yang, H., & Wang, Y. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105910 · Cited by 6
Almubayyidh, M., Alghamdi, I., Jenkins, D. A., & Parry‐Jones, A. (2025) · Healthcare · doi:10.3390/healthcare13080876 · Cited by 4
“Two reviewers used the Prediction Model Risk of Bias Assessment Tool (PROBAST) to evaluate the quality of the included studies [12].”
Impacto de la IA en la Predicción y Detección Oportuna de Enfermedades: Una Revisión Sistemática
Zepeda-Lugo, C., Rivarola, A. M. I., & Ferreira, A. P. A. (2024) · Latin american journal of applied engineering. · doi:10.69681/lajae.v7i1.35 · Cited by 6
“El riesgo de sesgo se evaluó utilizando la herramienta de Cochrane (RoB 2) [26] para ensayos clínicos aleatorizados y ROBINS-I [27] para intervenciones en estudios no aleatorizados y observacionales.”
Predicting gastric cancer survival using machine learning: A systematic review
Wang, H.-N., An, J.-H., Wang, F., Hu, W., & Zong, L. (2025) · World Journal of Gastrointestinal Oncology · doi:10.4251/wjgo.v17.i5.103804 · Cited by 9
“The risk of bias in the included prediction models was assessed by two reviewers using the Prediction Model Risk of Bias Assessment Tool (PROBAST)[18].”
Oral and Maxillofacial Low-Grade Myofibroblastic Sarcoma: A Systematic Review
Giraldo‐Roldán, D., de Paiva, J. P. G., Roza, A. L. O. C., Louredo, B. V. R., Santos-Leite, É. G., & Vargas, P. A. (2025) · Head and Neck Pathology · doi:10.1007/s12105-025-01783-1 · Cited by 4
“…Guidance for Defining Review Questions (CHARMS Checklist) and the Guide for Systematic Reviews and Meta-analyses of Observational Studies (PMSs) [15, 16] in which P articipants/ P opulations are patients diagnosed with low-grade myofibroblastic sarcoma in the oral and maxil - lofacial region, E…”
Husile, H., Bao, Q., Sarula, S., La, C., Wujisiguleng, W., Siqintu, S., & Temuqile, T. (2025) · Brain and Behavior · doi:10.1002/brb3.70557 · Cited by 6
Prediction Models for Falls Risk Among Inpatients: A Systematic Review and Meta‐Analysis
Zhao, G., Zhang, Y., Luo, J., Tong, Y., & Sui, W. (2025) · Journal of Advanced Nursing · doi:10.1111/jan.17079 · Cited by 5
Fácila, L., Vidal-Pérez, R., Benedito, M., Andreu, M. L. S., Cervantes, C. E., & Fort, A. C. (2025) · REC CardioClinics · doi:10.1016/j.rccl.2025.07.001 · Cited by 4
Younger, J., Morris, E., Arnold, N., Athulathmudali, C., Pinidiyapathirage, J. M., & MacAskill, W. (2025) · Japanese Journal of Radiology · doi:10.1007/s11604-025-01853-y · Cited by 4
“TRIPOD + AI [17] was used for quality appraisal and PROBAST [18] was used to assess the risk of bias.”
Machine learning in the prediction of diabetic peripheral neuropathy: a systematic review
Ma, Y., Wang, Z., Yao, Z. Q., Lu, B., & He, Y. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03201-6 · Cited by 9
“The Prediction model Risk of Bias ASsessment Tool (PROBAST) [18] was used to evaluate the quality of articles by two reviewers independently.”
Lin, Y., Xu, Z., Ding, X., Chen, L., & Dai, K. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-056510 · Cited by 5
Garcha, I., & Phillips, S. P. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-067638 · Cited by 9
Prognostic models for clinical outcomes in patients with venous leg ulcers: A systematic review
Veličković, V. M., Macmillan, T., Kottner, J., Crompton, A., Munro, I., Paine, A., Savović, J., Spelman, T. D., Clark, M., Smit, H. J., et al. (2023) · Journal of Vascular Surgery Venous and Lymphatic Disorders · doi:10.1016/j.jvsv.2023.06.017 · Cited by 5
van der Tuin, S., Booij, S. H., Muller, M. K., van den Berg, D. P. G., Oldehinkel, A. J., & Wigman, J. T. W. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115546 · Cited by 4
“However, we used the recommended strategy, i.e., multiple imputation, to deal with this (Moons et al., 2019).”
Yuan, T., Edelmann, D., Kather, J. N., Fan, Z., Tagscherer, K. E., Roth, W., Bewerunge‐Hudler, M., Brobeil, A., Kloor, M., Bläker, H., et al. (2023) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2023.104199 · Cited by 6
You, J., Fu, Y., Cai, H., Wang, X., Li, Y., Zhang, M., Tang, J., Gao, P., Cai, Y., & Peng, B. (2024) · Journal of Gastrointestinal Surgery · doi:10.1016/j.gassur.2024.01.006 · Cited by 4
Jiu, L., Wang, J., Somolinos-Simón, F. J., Tapia, J. A. R., García-Sáez, G., Hernando, M. E., Li, X., Vreman, R. A., Mantel-Teeuwisse, A. K., & Goettsch, W. (2024) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2024.111574 · Cited by 7
Zhang, Y. Y., Zhu, X., Gao, F., & Yang, S. (2024) · Risk Management and Healthcare Policy · doi:10.2147/rmhp.s451436 · Cited by 5
“18 Inconsistencies were resolved by discussion or with the assistance of the corresponding author.”
BackgroundSarrió‐Sanz, P., Martínez-Cayuelas, L., Beltrán-Pérez, A., Muñoz-Montoya, M., Segura, J., Gil-Guillén, V. F., & Gómez-Pérez, L. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13082177 · Cited by 4
“Finally, a blinded author assessed the risk of bias and applicability with the Prediction model study Risk Of Bias Assessment Tool (PROBAST) [22] and extracted relevant items from this study following the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling (CHARMS)…”
MethodBotelho, S. F., Pantuzza, L. L. N., & Reis, A. M. M. (2024) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2024.05.005 · Cited by 4
Predicting Recovery After Concussion in Pediatric Patients: A Meta-Analysis
Wyrwa, J. M., Hoffberg, A. S., Stearns‐Yoder, K. A., Lantagne, A., Kinney, A. R., Reis, D. J., & Brenner, L. A. (2024) · PEDIATRICS · doi:10.1542/peds.2023-065431 · Cited by 5
Systematic Review of Clinical Prediction Models for the Risk of Emergency Caesarean Births
Hunt, A., Bonnett, L. J., Heron, J. E., Lawton, M., Clayton, G., Smith, G. C. S., Norman, J. E., Kenny, L. C., Lawlor, D. A., Merriel, A., et al. (2024) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17948 · Cited by 6
Nijland, N., Su, N., Gerdes, V. E. A., & Loos, B. G. (2024) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.14069 · Cited by 8
“So, we focused on model calibration and discrimination instead; these provide more stable outcomes because they do not change by different cut-off values (Moons et al. 2019).”
Dretzke, J., Abou-Foul, A. K., Albon, E., Hillier, B., Scandrett, K., Price, M. J., Moore, D. J., Mehanna, H. M., & Nankivell, P. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-090393 · Cited by 7
“Further external validation of existing models to assess generalisability should be limited to those models including HPV status as a variable.”
Hou, Z., Li, X., Yang, L., Liu, T., Lv, H., & Sun, Q. (2025) · Aging Clinical and Experimental Research · doi:10.1007/s40520-024-02911-7 · Cited by 4
Raising awareness of potential biases in medical machine learning: Experience from a Datathon
Hochheiser, H., Klug, J., Mathie, T., Pollard, T., Raffa, J. D., Ballard, S. L., Conrad, E. A., Edakalavan, S., Joseph, A. M., Alnomasy, N., et al. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000932 · Cited by 4
Rayner, D. G., Shah, D. S., Dai, S.-C., Gou, D., Chen, J., Agarwal, A., Mustafa, R. A., Manja, V., Vandvik, P. O., Agoritsas, T., et al. (2025) · BMJ Medicine · doi:10.1136/bmjmed-2025-001369 · Cited by 5
“16 Disagreements were resolved through discussion between reviewers or, if necessary, adjudication by a third reviewer (FF).”
Metadata
- Authors
- Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S.
- Affiliations
- Utrecht University (NL)
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- W1-W33
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907638671
- Cited by
- 1,669
- References
- 201
- Citations by year
- 2026: 359 · 2025: 330 · 2024: 253 · 2023: 236 · 2022: 193 · 2021: 175 · 2020: 96 · 2019: 24
- FWCI
- 56.12
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (publishedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- PROBAST, risk of bias assessment, applicability, prediction model, diagnostic model, prognostic model, model validation
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.