PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration
Open accessThe DOI matches its Crossref record; 194 of the 201 references it cites were found in a registry or an index.
Abstract
Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMeasuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Research works printed but not linked: 2
- 10.1177/0962280218785504.30032705 · doi:10.1177/0962280218785504.30032705
- Wong SS, 2003, AMIA Annu Symp Proc, 728
Cited by
Items 401–500 of 1,003
Taylor, B., Barboi, C., & Boustani, M. A. (2023) · Journal of the American Geriatrics Society · doi:10.1111/jgs.18426 · Cited by 18
“According to the same task force, verification bias is present when only a subset of subjects receives the standard confirmation of disease status, and its likelihood depends on the diagnostic test result.(18)”
BackgroundZhang, X., Wang, X., Xu, L., Liu, J., Ren, P., & Wu, H. (2023) · European journal of medical research · doi:10.1186/s40001-023-01027-4 · Cited by 23
“We assessed the risk of bias in reported prediction models using PROBAST, risk of a bias assessment tool designed to systematically evaluate diagnostic or prognostic prediction models [16].”
MethodGonzalez, R., Nejat, P., Saha, A., Campbell, C. J. V., Norgan, A. P., & Lokker, C. (2023) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2023.100348 · Cited by 27
Johns, E., Godet, J., Alkanj, A., Beck, M., Mas, L. D., Gourieux, B., Sauleau, E., & Michel, B. (2023) · European Journal of Hospital Pharmacy · doi:10.1136/ejhpharm-2023-003857 · Cited by 28
Qian, L.-J., Wu, T., Kong, S., Lou, X., Jiang, Y., Tan, Z., Wu, L., & Gao, C. (2024) · European Journal of Radiology · doi:10.1016/j.ejrad.2024.111314 · Cited by 20
A Review on Predictive Systems and Data Models for COVID-19
Khan, F. N., Khanam, A., Ramlal, A., & Ahmad, S. (2020) · Studies in computational intelligence · doi:10.1007/978-981-15-8534-0_7 · Cited by 25
Brinza, C., Burlacu, A., Tinică, G., Covic, A. C., & Macovei, L. (2021) · Healthcare · doi:10.3390/healthcare9020148 · Cited by 20
Klaveren, D. V., Rekkas, A., Alsma, J., Verdonschot, R. J. C. G., Koning, D., Kamps, M. J. A., Dormans, T. P. J., Stassen, R. C., Weijer, S., Arnold, K.-S., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-051468 · Cited by 19
Tohidinezhad, F., Pennetta, F., van Loon, J., Dekker, A. L. A. J., Ruysscher, D. D., & Traverso, A. (2022) · Clinical and Translational Radiation Oncology · doi:10.1016/j.ctro.2022.02.007 · Cited by 23
Kokkinakis, S., Ziogas, I. A., Salazar, J. D. L., Moris, D., & Tsoulfas, G. (2024) · Cancers · doi:10.3390/cancers16091645 · Cited by 25
Number of Publications on New Clinical Prediction Models: A Bibliometric Review
Arshi, B., Wynants, L., Rijnhart, E., Reeve, K. A., Cowley, L., & Smits, L. J. M. (2025) · JMIR Medical Informatics · doi:10.2196/62710 · Cited by 28
Accuracy of Machine Learning Models to Predict In-hospital Cardiac Arrest: A Systematic Review
Moffat, L., & Xu, D. (2021) · Clinical Nurse Specialist · doi:10.1097/nur.0000000000000644 · Cited by 17
Risk Prediction Model for Esophageal Cancer Among General Population: A Systematic Review
Chen, R., Zheng, R., Zhou, J., Li, M., Shao, D., Li, X., Wang, S., & Wei, W. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.680967 · Cited by 19
Prediction models of vaginal birth after cesarean delivery: A systematic review
Deng, B., Li, Y., Chen, J., Guo, J., Tan, J., Yang, Y., & Liu, N. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2022.104359 · Cited by 20
Risk stratification in adult and pediatric pulmonary arterial hypertension: A systematic review
Lokhorst, C., van der Werf, S., Berger, R. M. F., & Douwes, J. M. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.1035453 · Cited by 18
Chen, F., Kantagowit, P., Nopsopon, T., Chuklin, A., & Pongpirul, K. (2023) · PLoS ONE · doi:10.1371/journal.pone.0278729 · Cited by 29
“Based on ROB results from the four domains, an overall ROB rating and prediction model applicability rating will be given to the prediction model following the recommendations in the PROBAST assessment form.”
BackgroundInfluentialVenn, M. L., Hooper, R., Pampiglione, T., Morton, D. G., Nepogodiev, D., & Knowles, C. H. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073085 · Cited by 23
Fink, H. A., Butler, M. E., Claussen, A. M., Collins, E. S., Krohn, K., Taylor, B. C., Tikabo, S. S., Vang, D., Zerzan, N. L., & Ensrud, K. E. (2023) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.4895 · Cited by 17
Moon, S. J., Lee, S., Hwang, J., Lee, J., Kang, S., & Cha, H. (2023) · RMD Open · doi:10.1136/rmdopen-2023-003783 · Cited by 19
Stroke risk prediction models: A systematic review and meta-analysis
Asowata, O. J., Okekunle, A. P., Olaiya, M. T., Akinyemi, J. O., Owolabi, M. O., & Akpa, O. M. (2024) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2024.122997 · Cited by 24
Foodborne Event Detection Based on Social Media Mining: A Systematic Review
Salaris, S., Ocagli, H., Casamento, A., Lanera, C., & Gregori, D. (2025) · Foods · doi:10.3390/foods14020239 · Cited by 23
“PROBAST is designed to assess RoB in diagnostic and prognostic prediction model studies and comprises four domains—participants, predictors, outcomes, and analysis—covering 20 signaling questions (Qs) [21].”
Palazón‐Bru, A., Mares‐García, E., López‐Bru, D., Mares‐Arambul, E., la Rosa, D. M. F.-D., Carbonell‐Torregrosa, M. D. L. Á., & Gil-Guillén, V. F. (2019) · Head & Neck · doi:10.1002/hed.26025 · Cited by 17
Development of a prehospital prediction model for risk stratification of patients with chest pain
Wibring, K., Lingman, M., Herlitz, J. D., Ashfaq, A., & Bång, A. (2021) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2021.09.079 · Cited by 23
Reuter-Selbach, M. J., Su, N., & Faggion, C. M. (2022) · Journal of Evidence Based Dental Practice · doi:10.1016/j.jebdp.2022.101793 · Cited by 17
Hernigou, P. (2022) · International Orthopaedics · doi:10.1007/s00264-022-05631-7 · Cited by 17
Zhang, H., Shao, J., Chen, D., Zou, P., Cui, N., Tang, L., Wang, D., & Ye, Z. (2020) · Diabetes Metabolic Syndrome and Obesity · doi:10.2147/dmso.s283949 · Cited by 19
Underwood, B. R., Lourida, I., Gong, J., Tamburin, S., Tang, E. Y. H., Sidhom, E., Tai, X. Y., Betts, M. J., Ranson, J. M., Zachariou, M., et al. (2025) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1002/trc2.70037 · Cited by 28
“22 The JBI tools used contain 10 questions considering sample selection, assessment of exposure and outcome measures, statistical analysis, and strategies to deal with confounders and loss to follow-up.”
van Haren, I. E. P. M., van der Worp, M. P., van Rijn, R. M., Stubbe, J. H., van Cingel, R. E. H., Verbeek, A. L. M., van der Wees, P. J., & Staal, J. B. (2025) · Annals of Physical and Rehabilitation Medicine · doi:10.1016/j.rehab.2024.101921 · Cited by 30
Kaul, T., Damen, J. A. A., Wynants, L., Van Calster, B., van Smeden, M., Hooft, L., & Moons, K. G. M. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111732 · Cited by 16
Self-citationWang, X., Zhang, P., Lu, H., Luo, D., Yang, D., Li, K., Qiu, S., Zeng, H., & Zeng, X. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-088253 · Cited by 26
“We assessed the risk of bias and suitability of the included studies using the Prediction Model Risk of Bias Assessment Tool (PROBAST), 16 which assesses the quality and potential risk of bias in the selected studies.”
Zhang, H., Chen, D., Shao, J., Zou, P., Cui, N., Tang, L., Wang, X., Wang, D., Wu, J., & Ye, Z. (2021) · Risk Management and Healthcare Policy · doi:10.2147/rmhp.s328180 · Cited by 16
Li, L., Li, X., Li, W., Ding, X., Zhang, Y., Chen, J., & Li, W. (2022) · BMC Cancer · doi:10.1186/s12885-022-09841-5 · Cited by 16
Fascial Dehiscence and Incisional Hernia Prediction Models: A Systematic Review and Meta‐analysis
Tansawet, A., Numthavaj, P., Techapongsatorn, T., Techapongsatorn, S., Attia, J., McKay, G. J., & Thakkinstian, A. (2022) · World Journal of Surgery · doi:10.1007/s00268-022-06715-6 · Cited by 16
Machine Learning in Predicting Tooth Loss: A Systematic Review and Risk of Bias Assessment
Hasuike, A., Watanabe, T., Wakuda, S., Kogure, K., Yanagiya, R., Byrd, K. M., & Sato, S. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12101682 · Cited by 24
“A summary of the PROBAST assessments is shown in Figure 2.”
BackgroundMethodInfluentialKaiser, I., Pfahlberg, A. B., Mathes, S., Uter, W. J. C., Diehl, K., Steeb, T., Heppt, M. V., & Gefeller, O. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12051976 · Cited by 19
Wang, S., Huang, H., Hou, M., Xu, Q., Qian, W., Tang, Y., Li, X., Qian, G., Ma, J., Zheng, Y., et al. (2023) · Pediatric Research · doi:10.1038/s41390-023-02558-6 · Cited by 18
“When the number of events per variable (EPVs) is too small and continuous variables are converted to dichotomous variables, univariate analysis or automatic forward and backward methods are used to screen variables, and the overfitting problem becomes more serious while developing prediction models.(15) External and i…”
MethodDevelopment and validation of prediction model for older adults with cognitive frailty
Huang, J., Zeng, X., Ning, H., Peng, R., Guo, Y., Hu, M., & Feng, H. (2024) · Aging Clinical and Experimental Research · doi:10.1007/s40520-023-02647-w · Cited by 17
Rey, R., Gualtieri, R., Scala, G. C. L., & Posfay-Barbe, K. M. (2024) · European Journal of Pediatric Surgery · doi:10.1055/a-2257-5122 · Cited by 19
“22”
Siegler, J. E., Koneru, M., Qureshi, M. M., Doheim, M. F., Nogueira, R. G., Martinez‐Majander, N., Nagel, S., Penckofer, M., Demeestere, J., Puetz, V., et al. (2024) · Journal of the American Heart Association · doi:10.1161/jaha.124.034948 · Cited by 18
Musto, L., Smith, A. J., Pepper, C., Bujkiewicz, S., & Bown, M. J. (2024) · British journal of surgery · doi:10.1093/bjs/znae239 · Cited by 19
“The risk of bias was summarized as low, high, or unclear for each domain, and the evaluation similarly repeated for the first three domains in regard to applicability, producing seven scores for each study 15 .”
MethodEssers, B., Gils, A. V., Lafosse, C., Michielsen, M., Beyens, H., Schillebeeckx, F., Veerbeek, J. M., Luft, A. R., Kos, D., & Verheyden, G. (2021) · BMC Neurology · doi:10.1186/s12883-021-02493-1 · Cited by 20
Naye, F., Décary, S., Houle, C., LeBlanc, A., Cook, C. E., Dugas, M., Skidmore, B., & Tousignant‐Laflamme, Y. (2023) · Physical Therapy · doi:10.1093/ptj/pzad021 · Cited by 20
Blythe, R. D., Parsons, R., Barnett, A. G., McPhail, S., & White, N. M. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.05.020 · Cited by 16
Jassal, K., Edwards, M., Koohestani, A., Brown, W. A., Serpell, J. W., & Lee, J. C. (2025) · Frontiers in Endocrinology · doi:10.3389/fendo.2025.1506729 · Cited by 18
Schmeltzer, P. A., & Russo, M. W. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10194476 · Cited by 18
Wang, Y., Lin, X., & Sun, D. (2021) · Annals of Translational Medicine · doi:10.21037/atm-21-4733 · Cited by 20
van den Brink, D. A., de Vries, I. S. A., Datema, M., Perot, L., Sommers, R., Daams, J. G., Calis, J. C. J., Brals, D., & Voskuijl, W. P. (2022) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2023.113448 · Cited by 19
Kohn, R. A., Weissman, G. E., Wang, W., Ingraham, N. E., Scott, S., Bayes, B. A., Anesi, G. L., Halpern, S. D., Kipnis, P., Liu, V. X., et al. (2023) · Medical Care · doi:10.1097/mlr.0000000000001878 · Cited by 19
Hu, Y., Lu, H. S., Ren, L., Yang, M., Shen, M., Huang, J., Huang, Q., & Fu, L. (2023) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2023.104546 · Cited by 17
Korosoglou, G., Sagris, Μ., André, F., Steen, H., Montenbruck, M., Frey, N., & Kelle, S. (2024) · Scientific Reports · doi:10.1038/s41598-023-50835-5 · Cited by 16
“Bias assessment was performed according to the PROBAST tool, which is specialized for assessing the bias and applicability of studies developing prediction models 21 .”
MethodLiu, Y., Li, R., Chen, L., Zhao, F., Su, Y., Jin, S., & Liu, J. (2024) · Journal of Cancer Survivorship · doi:10.1007/s11764-024-01566-7 · Cited by 15
“The risk-prediction model constructed by statistical analysis is the scientific method to confirm the quantitative relationship between multiple risk factors and the probability of the occurrence of outcomes [20].”
Prediction models for amputation after diabetic foot: systematic review and critical appraisal
Huang, J., Yang, J., Qi, H., Xu, M., Xu, X. S., & Zhu, Y. (2024) · Diabetology & Metabolic Syndrome · doi:10.1186/s13098-024-01360-6 · Cited by 20
Siriyotha, S., Tantrakul, V., Plitphonganphim, S., Rattanasiri, S., & Thakkinstian, A. (2021) · Diagnostics · doi:10.3390/diagnostics11061097 · Cited by 15
Hasan, M. K., Bath, P. A., Marincowitz, C., Sutton, L., Pilbery, R., Hopfgartner, F., Mazumdar, S., Campbell, R., Stone, T., Thomas, B., et al. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106024 · Cited by 21
Lv, X., Liu, D., Chen, X., Chen, L., Wang, X., Xu, X., Chen, L., & Huang, C. S. (2024) · BMC Infectious Diseases · doi:10.1186/s12879-024-10380-6 · Cited by 18
Validation of a Dynamic Risk Prediction Model Incorporating Prior Mammograms in a Diverse Population
Jiang, S., Bennett, D. L., & Colditz, G. A. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.12681 · Cited by 16
Dhahbi, W., Jebabli, N., Souaifi, M., Ceylan, H. İ., Saad, H. B., Chamari, K., Pyne, D. B., & Chaabène, H. (2026) · Bioengineering · doi:10.3390/bioengineering13060692 · Cited by 31
“Pre-registration of validation protocols and prospective evaluation designs would reduce reporting bias and selective outcome reporting [56,57].”
Clinical prediction models for aneurysmal subarachnoid hemorrhage: a systematic review update
Parekh, A., Satish, S., Dulhanty, L., Berzuini, C., & Patel, H. C. (2023) · Journal of NeuroInterventional Surgery · doi:10.1136/jnis-2023-021107 · Cited by 15
Kiss, B., Nagy, R., Kói, T., Harnos, A., Edes, I. F., Ábrahám, P., Mészáros, H., Hegyi, P., & Zima, E. (2024) · PLoS ONE · doi:10.1371/journal.pone.0293704 · Cited by 14
“The Prediction model Risk Of Bias ASsessment Tool (PROBAST) was used to assess the risk of bias and the applicability of primary studies following the recommendation of the Cochrane Collaboration [15].”
MethodBrogna, C., Montano, L., Zanolin, M. E., Bisaccia, D. R., Ciammetti, G., Viduto, V., Fabrowski, M., Baig, A. M., Gerlach, J., Gennaro, I., et al. (2024) · Journal of Medical Virology · doi:10.1002/jmv.29507 · Cited by 15
“…from the onset of first symptoms during the acute phase) plus probiotics Lactobacillus reuteri (10 million cfu, once daily for 20 days) and Bacillus clausii (8 million cfu, once daily for 20 days) or only the combination of probiotics for the others were taken by the 21 LC patients, according to.”
MethodMeng, L., & Ho, P. (2024) · The Journal of Vascular Access · doi:10.1177/11297298241237830 · Cited by 18
“22 The potential risk of bias and concern for applicability were assessed over four domains (Participants, Predictors, Outcome, and Analysis) and three domains (Participants, Predictors, and Outcome), respectively.”
Mäenpää, S. M., & Korja, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105523 · Cited by 22
Revisiting the use of adverse childhood experience screening in healthcare settings
Danese, A., Asmussen, K., MacLeod, J. G., Meehan, A. J., Sears, J., Slopen, N. B., Smith, P. A., & Sweeney, A. (2024) · Nature Reviews Psychology · doi:10.1038/s44159-024-00362-5 · Cited by 18
“Methodological guidance 100,101 indicates the necessary steps for accurate prediction modelling of ACE-based individual risk for health problems.”
Risk prediction models for disability in older adults: a systematic review and critical appraisal
Zhou, J., Xu, Y., Yang, D., Zhou, Q., Ding, S., & Pan, H. (2024) · BMC Geriatrics · doi:10.1186/s12877-024-05409-z · Cited by 18
“For handling missing values, two studies [16, 18] used multiple imputation, which is considered the most appropriate approach [32].”
MethodInfluentialPost-Cardiac arrest outcome prediction using machine learning: A systematic review and meta-analysis
Zobeiri, A., Rezaee, A., Hajati, F., Argha, A., & Alinejad‐Rokny, H. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105659 · Cited by 23
Li, Z., Yin, S., Wang, S., Wang, Y., Qiang, W., & Jiang, J. (2024) · Journal of Advanced Research · doi:10.1016/j.jare.2024.11.018 · Cited by 18
“Two authors (S.W. and S.Y.) independently evaluated the methodological quality of all peer-reviewed studies (N = 162) included using the Prediction Model Study Risk of Bias Assessment Tool (PROBAST) tool [34].”
Lord, S. J., Horvath, A. R., Sandberg, S., Monaghan, P. J., Cobbaert, C. M., Reim, M., Tolios, A., Mueller, R., & Bossuyt, P. M. (2025) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2025.2453148 · Cited by 18
“Specific guides are available for appraising risk of bias and applicability for published studies of diagnostic [9] and prognostic tests [56] and prediction models [57].”
López-Jiménez, N., Sánchez, F. G., Pailos, R. H., Rodrigo-Álvaro, V., Pascual-Pedreño, A. I., Moreno‐Cid, M., Delgado‐Rodríguez, M., & Hernández‐Martínez, A. (2021) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16947 · Cited by 17
Hu, M., Gao, Y., Kwok, T. C. Y., Shao, Z., Xiao, L. D., & Feng, H. (2022) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2022.755005 · Cited by 15
Meijs, D. A. M., van Kuijk, S. M. J., Wynants, L., Stessel, B., Mehagnoul-Schipper, J., Hana, A., Scheeren, C. I. E., Bergmans, D. C. J. J., Bickenbach, J., Laenen, M. V., et al. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.10.015 · Cited by 16
“[3] using the Prediction model study Risk Of Bias Assessment Tool (PROBAST) [15].”
Predicting Hypoxia Using Machine Learning: Systematic Review
Pigat, L., Geisler, B. P., Sheikhalishahi, S., Sander, J., Kaspar, M., Schmutz, M., Rohr, S. O., Wild, C. M., Goss, S., Zaghdoudi, S., et al. (2024) · JMIR Medical Informatics · doi:10.2196/50642 · Cited by 15
“To assess the risk of bias, quality, and applicability of the studies included, Prediction Model Risk of Bias Assessment Tool (PROBAST) was used [14].”
MethodCooray, S. D., Boyle, J. A., Soldatos, G., Zamora, J., Fernandez-Felix, B. M., Allotey, J., Thangaratinam, S., & Teede, H. J. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-038845 · Cited by 14
Ahmeidat, A., Kotts, W. J., Wong, J., McLernon, D. J., & Black, M. E. (2021) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2021.05.011 · Cited by 14
Development of risk models of incident hypertension using machine learning on the HUNT study data
Schjerven, F. E., Ingeström, E., Steinsland, I., & Lindseth, F. (2024) · Scientific Reports · doi:10.1038/s41598-024-56170-7 · Cited by 20
“Third, with the increased capacity of many ML methods for learning data patterns, ensuring proper internal validation during model development is important 50 , which has been lacking for hypertension risk models 7 .”
Prediction Pathway for Severe Asthma Exacerbations
Yadav, C. P., Chakraborty, A., Price, D. B., Lim, L. H. M., Juang, Y. R., Beasley, R. W., Sadatsafavi, M., Janson, C., Koh, M. S., Wang, E., et al. (2025) · CHEST Journal · doi:10.1016/j.chest.2025.04.046 · Cited by 16
“14 Besides, to further understand how the predictors interact in leading to an asthma exacerbation, it is imperative to delineate their interactions to uncover their speci fi c roles in the core prediction pathway.”
Wang, S., Wang, K., & Lin, J. (2025) · Scientific Reports · doi:10.1038/s41598-025-05505-z · Cited by 19
Abesi, F., Maleki, M., & Zamani, M. (2023) · Imaging Science in Dentistry · doi:10.5624/isd.20220224 · Cited by 13
“The risk of bias of the enrolled studies was examined using the adapted criteria of the Prediction Model Risk of Bias Assessment Tool (PROBAST).(17) Based on PROBAST, the studies were rated for risk of bias and concerns about applicability as low, high, or unclear.”
MethodHernigou, P., Verrier, S., Homma, Y., Rouard, H., Flouzat–Lachaniette, C.-H., & Kumar, K. H. S. (2023) · International Orthopaedics · doi:10.1007/s00264-023-05788-9 · Cited by 13
“The input layer receives data and other elements, and normalizes them into units that may be used in the hidden layer’s mathematical operations [16, 17].”
BackgroundTiberio, P., Gaudio, M., Belloni, S., Pindilli, S., Benvenuti, C., Jacobs, F., Saltalamacchia, G., Zambelli, A., Santoro, A., & Sanctis, R. D. (2023) · Cancers · doi:10.3390/cancers15133424 · Cited by 15
“The PROBAST (Prediction model Risk of Bias Assessment Tool) tool was utilized to assess the risk of bias (ROB) and the applicability of predictive model research [38,39].”
MethodVilliger, R., Julliard, P., Farhoumand, P. D., Choffat, D., Tritschler, T., Stalder, O., Rossel, J., Aujesky, D., Méan, M., & Baumgartner, C. (2023) · Thrombosis Research · doi:10.1016/j.thromres.2023.08.003 · Cited by 19
Hua, Y., Yuan, Y., Wang, X., Liu, L., Zhu, J., Li, D., & Tu, P. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1226473 · Cited by 21
“In terms of sample size, PROBAST requires that model development studies should have more than 20 events per variable (EPV) to avoid overfitting of the model; model validation studies should include at least 100 subjects with outcomes (40).”
MethodByrne, J. F., Mongan, D., Murphy, J., Healy, C., Föcking, M., Cannon, M. D., & Cotter, D. (2023) · Translational Psychiatry · doi:10.1038/s41398-023-02623-y · Cited by 15
“This systematic review was prospectively registered with PROS-PERO, CRD42022302047.”
BackgroundMethodMultivariable prediction models of caries increment: a systematic review and critical appraisal
Havsed, K., Petersson, G. H., Isberg, P., Pigg, M., Svensäter, G., & Rohlin, M. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02298-y · Cited by 20
“For the assessment of risk of bias and applicability, which are essential steps in any SR, the Prediction model Risk Of Bias ASsessment Tool (PROBAST) was developed [5, 6].”
BackgroundMethodInfluentialAbesi, F., Hozuri, M., & Zamani, M. (2023) · Journal of Clinical and Experimental Dentistry · doi:10.4317/jced.60287 · Cited by 17
“As per the PROBAST, we rate the studies for the risk of bias and concerns about applicability as low, high, or unclear.”
MethodFu, Y., Feller, D., Koes, B. W., & Chiarotto, A. (2024) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2024.12081 · Cited by 17
SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models
Navarro, C. L. A., Damen, J. A., Ghannad, M., Dhiman, P., van Smeden, M., Reitsma, J. B., Collins, G. S., Riley, R. D., Moons, K. G. M., & Hooft, L. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111364 · Cited by 16
Self-citationOakley, W., Tandle, S., Perkins, Z., & Marsden, M. E. R. (2024) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000004385 · Cited by 17
“51 Predictive performance reporting was not standardized, and only two models reported measures of model calibration.”
Moharrami, M., Zavareh, P. A., Watson, E. E., Singhal, S., Johnson, A. E. W., Hosni, A., Quiñonez, C., & Glogauer, M. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307531 · Cited by 15
“However, for prediction models developed without external validation, even if all domains were rated low, the risk of bias should be considered high unless the model was developed on a very large dataset and included internal validation [18].”
MethodFares, M. Y., Liu, H. H., Etges, A. P. B. D. S., Zhang, Y. B., Warner, J. J. P., Olson, J. J., Fedorka, C. J., Khan, A. Z., Best, M. J., Kirsch, J. M., et al. (2024) · JBJS Reviews · doi:10.2106/jbjs.rvw.24.00075 · Cited by 15
Huang, J., Chen, J., Yang, J., Han, M., Xue, Z., Wang, Y., Xu, M., Qi, H., & Wang, Y. (2024) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2024.103808 · Cited by 17
Prediction models for postpartum stress urinary incontinence: A systematic review
Wang, L., Zhang, M., Sha, K., Qiao, Y., & Dong, Q. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e37988 · Cited by 14
“Outcomes should be measured without knowledge of predictive information; some studies did not consider blinding when determining outcomes and collecting predictors, and the predictive power of models may be overestimated when outcomes are measured without blinding [34].”
BackgroundMethodResultInfluentialShi, L., Zhao, J., Wei, Z., Wu, H., & Sheng, M. (2024) · Frontiers in Oncology · doi:10.3389/fonc.2024.1381217 · Cited by 14
Sankar, H., Alagarsamy, R., Lal, B., Rana, S. S., Roychoudhury, A., Agrawal, A., & Wankhar, S. (2024) · Oral Surgery Oral Medicine Oral Pathology and Oral Radiology · doi:10.1016/j.oooo.2024.09.010 · Cited by 17
Chen, H., Yu, D., Zhang, J., & Li, J. (2024) · Clinical Therapeutics · doi:10.1016/j.clinthera.2024.09.013 · Cited by 18
Battle, C. E., Cole, E. M., Carter, K., & Baker, E. J. (2024) · BMC Emergency Medicine · doi:10.1186/s12873-024-01107-6 · Cited by 16
“Risk of bias and applicability were assessed using the “Prediction model Risk Of Bias ASsessment Tool” (PRO-BAST) [14] where: “Risk of bias refers to the extent that flaws in the design, conduct, and analysis of the primary prediction modelling study lead to biased, often overly optimistic,…”
MethodBeard, K., Pennington, A. M., Gauff, A. K., Mitchell, K. C., Smith, J. M., & Marion, D. W. (2024) · Biomedicines · doi:10.3390/biomedicines12112459 · Cited by 57
Vision-Based Collision Warning Systems with Deep Learning: A Systematic Review
Chitraranjan, C. D., Vipulananthan, V., & Sritharan, T. (2025) · Journal of Imaging · doi:10.3390/jimaging11020064 · Cited by 24
“We conducted a risk of bias analysis (ROB) for each included study using the PROBAST tool [19], which we adapted to the type of studies reviewed here (since the input to the studies reviewed here are image sequences without any predefined predictors, questions about predictor variables were…”
Clinical management of COVID-19: living guideline, June 2025
World Health Organization (2025) · World Health Organization eBooks · doi:10.2471/b09467 · Cited by 62
Systematic Review of Prediction Models for Preterm Birth Using CHARMS
Kim, J., & Lee, J. Y. (2021) · Biological Research For Nursing · doi:10.1177/10998004211025641 · Cited by 14
Prediction Model Risk-of-Bias Assessment Tool for coronary artery lesions in Kawasaki disease
Huang, H., Dong, J., Wang, S., Shen, Y., Zheng, Y., Jiang, J., Zeng, B., Li, X., Yang, F., Ma, S., et al. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.1014067 · Cited by 13
“Calibration and discrimination are often used to measure prediction model performance, and classical indicators such as sensitivity and specificity are used to evaluate the model classification abilities (10).”
BackgroundMethodInfluential
Metadata
- Authors
- Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S.
- Affiliations
- Utrecht University (NL)
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- W1-W33
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907638671
- Cited by
- 1,669
- References
- 201
- Citations by year
- 2026: 359 · 2025: 330 · 2024: 253 · 2023: 236 · 2022: 193 · 2021: 175 · 2020: 96 · 2019: 24
- FWCI
- 56.12
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (publishedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- PROBAST, risk of bias assessment, applicability, prediction model, diagnostic model, prognostic model, model validation
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.