PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · cited 3×
“A domain-based structure was adopted similar to that used in other risk of bias tools such as the revised Cochrane Risk of Bias tool,(29) QUADAS-2,(36) ROBINSI(30) and ROBIS.”
BackgroundMethodSelf-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · cited 3×
“A domain-based structure was adopted similar to that used in other risk of bias tools such as the revised Cochrane Risk of Bias tool,(29) QUADAS-2,(36) ROBINSI(30) and ROBIS.”
InfluentialSelf-citationCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147 · cited 11×
“A multivariable prediction model is defined as any combination or equation of two or more predictors for estimating the probability or risk for an individual.(7, 8, 33-35)”
InfluentialSelf-citation2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.11.005 · Cited by 4,945
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Circulation · doi:10.1161/01.cir.0000437741.48606.98 · Cited by 3,697
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
Self-citationAssessing Bias in Studies of Prognostic Factors
Hayden, J. A., van der Windt, D. A., Cartwright, J. L., Côté, P., & Bombardier, C. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-4-201302190-00009 · Cited by 3,287 · cited 1×
“(25, 26) The QUIPS tool has been developed to assess the risk of bias in predictor finding (prognostic factor) studies.(28) The methodological quality of studies investigating the impact of a prediction model using a comparative randomised design can be assessed using the revised Cochrane risk of bias tool (ROB 2.”
MethodCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
Self-citationSystematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Cited by
Items 801–900 of 1,010
Tayfur, A., Salles, J. I., Miller, S., Screen, H. R. C., & Morrissey, D. (2020) · Physical Therapy in Sport · doi:10.1016/j.ptsp.2020.05.004 · Cited by 17
Accuracy of Machine Learning Models to Predict In-hospital Cardiac Arrest: A Systematic Review
Moffat, L., & Xu, D. (2021) · Clinical Nurse Specialist · doi:10.1097/nur.0000000000000644 · Cited by 17
Bieri, U., Scharl, M. M., Sigg, S., Szczerba, B. M., Morsy, Y., Rüschoff, J. H., Schraml, P., Krauthammer, M. O., Hefermehl, L. J., Eberli, D., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-061421 · Cited by 15
Diagnostic models predicting paediatric viral acute respiratory infections: a systematic review
Rankin, D. A., Peetluk, L. S., Deppen, S. A., Slaughter, J. C., Katz, S. E., Halasa, N., & Khankari, N. K. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-067878 · Cited by 15
Kidney stones in renal transplant recipients: A systematic review
Piana, A., Basile, G., Masih, S., Bignante, G., Uleri, A., Gallioli, A., Prudhomme, T., Boissier, R., Pecoraro, A., Campi, R., et al. (2023) · Actas Urológicas Españolas (English Edition) · doi:10.1016/j.acuroe.2023.08.003 · Cited by 15
Zhu, M., Gao, J., Han, F., Yin, L.-L., Zhang, L., Yang, Y. S., & Zhang, J. (2023) · Insights into Imaging · doi:10.1186/s13244-023-01486-7 · Cited by 19
“And the degree of overfitting can vary widely depending on the type of internal validation techniques employed, such as cross-validation or bootstrapping, leading to biased or over-optimistic performance results [15].”
BackgroundMethodInfluentialKeshtkar, K., Safarpour, A. R., Heshmat, R., Sotoudehmanesh, R., & Keshtkar, A. (2023) · The Turkish Journal of Gastroenterology · doi:10.5152/tjg.2023.22491 · Cited by 16
Aghakhani, A., Yousefi, M., & Yekaninejad, M. S. (2023) · Annals of Otology Rhinology & Laryngology · doi:10.1177/00034894231206902 · Cited by 15
“23 None of the studies performed external validation.”
BackgroundRey, R., Gualtieri, R., Scala, G. C. L., & Posfay-Barbe, K. M. (2024) · European Journal of Pediatric Surgery · doi:10.1055/a-2257-5122 · Cited by 19
“The risk of bias determined using the PROBAST tool is shown in ► Fig.”
InfluentialParsaei, M., Arvin, A., Taebi, M., Seyedmirzaei, H., Cattarinussi, G., Sambataro, F., Pigoni, A., Brambilla, P., & Delvecchio, G. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1384828 · Cited by 17
Rayner, D. G., Kim, B. Y., & Foroutan, F. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111360 · Cited by 16
Development and Validation of an Automated, Real-Time Predictive Model for Postpartum Hemorrhage
Ende, H. B., Domenico, H. J., Polić, A., Wesoloski, A., Zuckerwise, L. C., McCoy, A. B., Woytash, A., Moore, R. P., & Byrne, D. W. (2024) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000005600 · Cited by 15
Oakley, W., Tandle, S., Perkins, Z., & Marsden, M. E. R. (2024) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000004385 · Cited by 17
Hoffman, H., Sims, J. J., Inoa‐Acosta, V., Hoit, D. A., Arthur, A. S., Draytsel, D. Y., Kim, Y. S., & Goyal, N. (2024) · Journal of NeuroInterventional Surgery · doi:10.1136/jnis-2024-021759 · Cited by 15
Alobaida, M., Joddrell, M., Zheng, Y., Lip, G. Y. H., Rowe, F. J., El‐Bouri, W. K., Hill, A. M., Lane, D. A., & Harrison, S. L. (2024) · Journal of the American Heart Association · doi:10.1161/jaha.123.033298 · Cited by 15
“Prediction Model Risk of Bias Assessment Tool contains 20 questions categorized into 4 domains: participants, predictors, outcome, and analysis to identify shortcomings and design flaws of a study.”
MethodDefining Levels of US Hospitals’ Pediatric Capabilities
Michelson, K. A., Alpern, E. R., Remick, K., Cash, R. E., Kemal, S., Wolk, C. B., Camargo, C. A., & Samuels‐Kalow, M. E. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.22196 · Cited by 15
Moharrami, M., Zavareh, P. A., Watson, E. E., Singhal, S., Johnson, A. E. W., Hosni, A., Quiñonez, C., & Glogauer, M. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307531 · Cited by 15
“The systematic review assessed the quality of the included studies using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) [17].”
Predicting psoriasis severity using machine learning: a systematic review
McMullen, E. P., Al-Naser, Y., Maazi, M., Grewal, R. S., Hafeez, D. A., Folino, T. R., & Vender, R. B. (2024) · Clinical and Experimental Dermatology · doi:10.1093/ced/llae348 · Cited by 18
Revisiting the use of adverse childhood experience screening in healthcare settings
Danese, A., Asmussen, K., MacLeod, J. G., Meehan, A. J., Sears, J., Slopen, N. B., Smith, P. A., & Sweeney, A. (2024) · Nature Reviews Psychology · doi:10.1038/s44159-024-00362-5 · Cited by 18
“Methodological guidance 100,101 indicates the necessary steps for accurate prediction modelling of ACE-based individual risk for health problems.”
Guo, H., Xu, K., Deng, F., Chen, Q., Liang, J., & Zhang, K. (2024) · Clinical and Applied Thrombosis/Hemostasis · doi:10.1177/10760296241285565 · Cited by 15
“Future researchers are encouraged to refer to PROBAST during the model-building process and adhere to reporting guidelines for multivariate forecasting models to minimize bias and ensure scienti fi c rigor in model development.”
MethodInfluentialKaul, R. D., Sacks, P., Thiel, C., Rimmer, J., Kalish, L. H., Campbell, R. G., Sacks, R., Ieva, A. D., & Harvey, R. J. (2024) · American Journal of Rhinology and Allergy · doi:10.1177/19458924241304082 · Cited by 16
Genetic biomarkers and machine learning techniques for predicting diabetes: systematic review
Khan, S., Mohsen, F., & Shah, Z. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11020-w · Cited by 20
“Additionally, hold-out validation compromises the data samples for model developing and may not effectively utilize avail - able data (Wolff, et al. 2019; Collins et al. 2015 ).”
Gouravani, M., Farahani, M. S., Salehi, M. A., Shojaei, S., Mirakhori, S., Harandi, H., Mohammadi, S., & Saleh, R. (2025) · BMC Cancer · doi:10.1186/s12885-025-13547-9 · Cited by 17
“Furthermore, we applied the Prediction Model Study Risk of Bias Assessment Tool (PROBAST) checklist to assess the presence of bias and evaluate the applicability of the included articles in four domains of participants, predictors, outcomes, and analyses (Table E3) [21].”
Fung, M. Y., Wong, Y. L., Cheung, K. M., Bao, K. H. K., & Sung, W. W. Y. (2025) · BMC Palliative Care · doi:10.1186/s12904-025-01696-4 · Cited by 18
“Risk of bias were assessed with PROBAST [20].”
Juang, Y. R., Ang, L., & Seow, W. J. (2025) · Scientific Reports · doi:10.1038/s41598-024-83875-6 · Cited by 18
“PROBAST provides a checklist of four domains: participants, predictors, outcome, and analysis, with signaling questions which are answered with “yes” (Y), “probably yes” (PY), “no” (N), “probably no” (PN) or “no information” (NI).”
InfluentialJassal, K., Edwards, M., Koohestani, A., Brown, W. A., Serpell, J. W., & Lee, J. C. (2025) · Frontiers in Endocrinology · doi:10.3389/fendo.2025.1506729 · Cited by 18
“The Prediction model Risk Of Bias Assessment Tool (PROBAST), used to evaluate the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was used to assess the included studies (32, 33).”
Horsfall, L., Bondaronek, P., Ive, J., & Poduval, S. (2025) · Clinical Epidemiology · doi:10.2147/clep.s527000 · Cited by 18
“95 Ensure Transparency Through Open Sharing Improve transparency by sharing code and de-identified datasets with permanent identifiers to allow for independent validation, bias assessment, and reproducibility.”
Chalkou, K., Steyerberg, E. W., Bossuyt, P. M., Subramaniam, S., Benkert, P., Kühle, J., Disanto, G., Kappos, L., Zecca, C., Egger, M., et al. (2021) · Diagnostic and Prognostic Research · doi:10.1186/s41512-021-00106-6 · Cited by 15
Siriyotha, S., Tantrakul, V., Plitphonganphim, S., Rattanasiri, S., & Thakkinstian, A. (2021) · Diagnostics · doi:10.3390/diagnostics11061097 · Cited by 15
“We followed a recently published framework for the meta-analysis of prediction models [23,24].”
MethodMontalto, C., Munafò, A. R., Arzuffi, L., Casula, M., Mirizzi, A. M., Costa, F., Leonardi, S., Visconti, L. O., & Ferlini, M. (2022) · Hellenic Journal of Cardiology · doi:10.1016/j.hjc.2022.04.008 · Cited by 15
Nelson, A. E., & Arbeeva, L. S. (2022) · The Journal of Rheumatology · doi:10.3899/jrheum.220326 · Cited by 26
“…of a multivariable prediction model for Individual Prognosis or Diagnosis), 59,60 or critical appraisal tools like PROBAST (Prediction model Risk of Bias Assessment Tool) 61 may be useful to avoid, address, and identify such potential biases, and are required by many peer-reviewed journals.”
BackgroundBlack, N., Henderson, I., Wattar, B. H. A., & Quenby, S. M. (2022) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000004940 · Cited by 15
Cheng, D., Liu, D., Philpotts, L. L., Turner, D. P., Houle, T. T., Chen, L. L., Zhang, M., Yang, J., Zhang, W., & Deng, H. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030482 · Cited by 18
“37 It includes 20 signalling questions across four domains (participants, predictors, outcome and analysis).”
BackgroundArtificial intelligence, machine learning and mental health
Delgadillo, J., & Atzil–Slonim, D. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-91497-0.00177-6 · Cited by 16
Hunik, L., Chaabouni, A., van Laarhoven, T., Hartman, T. O., Leijenaar, R. T. H., Cals, J., Uijen, A. A., & Schers, H. (2025) · JMIR Medical Informatics · doi:10.2196/62862 · Cited by 21
“Another reason for the high risk of bias was a nonappropriate data source that was used in 1 study [20] because the authors described the study as a case-control study although the study was not nested as recommended in the PROBAST guidelines [69,70].”
Jayasinghe, H. M., Wong, K. W., & Nugaliyadde, A. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112122 · Cited by 28
Post-stroke depression risk prediction models in stroke patients: A systematic review
Zeng, X., & Chen, X. J. (2025) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2025.07.003 · Cited by 14
Yu, M.-Y., Yoo, H. Y., Han, G. I., Kim, E. E., & Son, Y. (2025) · Journal of Medical Internet Research · doi:10.2196/76215 · Cited by 15
“The risk of bias and clinical applicability of each included study was evaluated using the PRO-BAST, which comprises 4 domains and 20 signal questions to evaluate bias and applicability [14].”
Lee, A., Lee, H., & Lee, S. H. (2025) · Medicina · doi:10.3390/medicina61111926 · Cited by 18
“…included studies was assessed according to the study design using the Risk of Bias 2 (RoB 2.0) tool [21], the Mixed Methods Appraisal Tool (MMAT) [22], the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool [23], and the Prediction model Risk of Bias Assessment Tool (PROBAST) [24].”
InfluentialKaiser, I., Mathes, S., Pfahlberg, A. B., Uter, W. J. C., Berking, C., Heppt, M. V., Steeb, T., Diehl, K., & Gefeller, O. (2022) · Cancers · doi:10.3390/cancers14123033 · Cited by 14
“According to the given rules in the PROBAST tool [13], the overall ROB is obtained by taking the lowest rating of any domain-specific ROB (“worst score counts principle”).”
MethodInfluentialPrediction models of diabetes complications: a scoping review
Ndjaboué, R., Ngueta, G., Rochefort-Brihay, C., Delorme, S., Guay, D., Ivers, N. M., Shah, B. R., Straus, S. E., Yu, C. H. Y., Comeau, S., et al. (2022) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2021-217793 · Cited by 16
Kooper, C. C., van der Zee, C. W., Oosterlaan, J., Plötz, F. B., & Königs, M. (2022) · Journal of Neurotrauma · doi:10.1089/neu.2022.0369 · Cited by 14
“33 In line with PROBAST guidelines, risk of bias was defined as shortcomings in the study design, conduct or analysis that can lead to systematically distorted estimates of the predictive performance.”
InfluentialMachine Learning and Artificial Intelligence in Surgical Research
Srinivas, S., & Young, A. J. (2023) · Surgical Clinics of North America · doi:10.1016/j.suc.2022.11.002 · Cited by 16
Gokhale, S., Taylor, D., Gill, J., Hu, Y., Zeps, N., Lequertier, V., Teede, H. J., & Enticott, J. (2023) · Digital Health · doi:10.1177/20552076231177497 · Cited by 16
“Using PROBAST,(28) studies are rated as low, moderate, and high concern for bias and applicability in each of four domains—participants, predictors, outcomes, and analysis.(18,27) We used guidance from the adaptation of the PROBAST tool for ML models reported elsewhere.”
MethodEvaluating Pelvic Tilt Using the Pelvic Antero-Posterior Projection Images: A Systematic Review
Chai, Y., Boudali, A. M., Khadra, S. M., Dasgupta, A., Maes, V., & Walter, W. L. (2023) · The Journal of Arthroplasty · doi:10.1016/j.arth.2023.10.035 · Cited by 14
Kooi, K., Talavera, E., Freundt, L. A., Oflazoglu, K., Ritt, M. J. P. F., Eberlin, K. R., Selles, R. W., Clemens, M. W., & Rakhorst, H. A. (2024) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000011266 · Cited by 15
“12”
Issanov, A., Aravindakshan, A., Puil, L. J., Tammemägi, M. C., Lam, S. C., & Dummer, T. (2024) · Diagnostic and Prognostic Research · doi:10.1186/s41512-024-00166-4 · Cited by 14
“This tool was specifically designed for assessing the risk of bias (ROB) and applicability of studies that develop, validate, or update prognostic prediction models [48].”
MethodPredicting Male Infertility Using Artificial Neural Networks: A Review of the Literature
Arroyo, V. S., Iosa, M., Antonucci, G., & Bartolo, D. D. (2024) · Healthcare · doi:10.3390/healthcare12070781 · Cited by 16
“The RoB analysis was performed through the administration of the prediction model risk of bias assessment tool (PROBAST) [36].”
MethodLi, J., Dan, K., & Ai, J. (2024) · Frontiers in Immunology · doi:10.3389/fimmu.2024.1281940 · Cited by 21
Fares, M. Y., Liu, H. H., Etges, A. P. B. D. S., Zhang, Y. B., Warner, J. J. P., Olson, J. J., Fedorka, C. J., Khan, A. Z., Best, M. J., Kirsch, J. M., et al. (2024) · JBJS Reviews · doi:10.2106/jbjs.rvw.24.00075 · Cited by 15
Chandramohan, D., Garapati, H. N., Nangia, U., Simhadri, P. K., Lapsiwala, B. J., Jena, N. K., & Singh, P. K. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1447057 · Cited by 16
Mortality Risk Prediction Models for People With Kidney Failure: A Systematic Review
Jarrar, F., Pasternak, M., Harrison, T. G., James, M. T., Quinn, R. R., Lam, N. N., Donald, M., Elliott, M. J., Lorenzetti, D., Strippoli, G. F., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.53190 · Cited by 18
Jiang, Y., Yan, H., Cui, J., Yang, K., & An, Y. (2025) · Helicobacter · doi:10.1111/hel.70026 · Cited by 14
Latt, P. M., Soe, N. N., Fairley, C. K., Chow, E. P. F., Johnson, C., Shah, P., Maatouk, I., Zhang, L., & Ong, J. J. (2025) · International Journal of Infectious Diseases · doi:10.1016/j.ijid.2025.107922 · Cited by 16
Kuo, K.-M., & Chang, C. S. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03010-x · Cited by 21
“This study assessed the risk of bias and applicability of the evidence based on the Prediction model risk of bias assessment tool (PROBAST) [106, 107].”
Mawdsley, E., Reynolds, B., & Cullen, B. (2021) · Journal of Neuropsychology · doi:10.1111/jnp.12244 · Cited by 14
“Further common sources of ROB came from excluding people for missing the outcome of interest in predictive models which can introduce bias if missing not at random (Wolff et al., 2019).”
BackgroundMethodInfluentialSystematic Review of Prediction Models for Preterm Birth Using CHARMS
Kim, J., & Lee, J. Y. (2021) · Biological Research For Nursing · doi:10.1177/10998004211025641 · Cited by 14
“Another three studies included mental illness as a predictor of medical status (Baer et al., 2018; Luo et al., 2016; Woolery, 1992).”
BackgroundResultInfluentialAppadurai, V., D’Elia, N. W., Mew, T. S.-H., Tomlinson, S., Chan, J. H., Hamilton-Craig, C. R., & Scalia, G. M. (2021) · IJC Heart & Vasculature · doi:10.1016/j.ijcha.2021.100849 · Cited by 14
López-Jiménez, N., Sánchez, F. G., Pailos, R. H., Rodrigo-Álvaro, V., Pascual-Pedreño, A. I., Moreno‐Cid, M., Delgado‐Rodríguez, M., & Hernández‐Martínez, A. (2021) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16947 · Cited by 17
Wang, Y., Lin, X., & Sun, D. (2021) · Annals of Translational Medicine · doi:10.21037/atm-21-4733 · Cited by 20
“PROBAST is a prediction model of bias assessment tool for assessing the risk of bias (ROB) and has applicability to review prediction models (156).”
MethodGeoghegan, L., Man, J., Jain, A., Price, A. J., Gibbons, E. J., Jerosch‐Herold, C., Sidey-Gibbons, C. J., & Rodrigues, J. N. (2021) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000008420 · Cited by 17
Jemioło, P., Storman, D., Mamica, M., Szymkowski, M., Żabicka, W., Wojtaszek-Główka, M., & Ligęza, A. (2022) · Sensors · doi:10.3390/s22072538 · Cited by 19
“Participants [73] 3.”
MethodZhong, M., van Zoest, V., Bilal, A.-M., Papadopoulos, F. C., & Castellano, G. (2022) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3536221.3556605 · Cited by 17
Susanty, S., Sufriyana, H., Su, E. C., & Chuang, Y. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280330 · Cited by 14
“This study followed the guidelines for developing and reporting machine learning predictive models in biomedical research [54] (see S1 Table in S1 File) and the prediction model risk of bias assessment tool (PROBAST) [55] (see S2 Table in S1 File).”
MethodInfluentialDabbagh, R., Jamal, A. A., Masud, J. H. B., Titi, M. A., Amer, Y. S., Khayat, A., Alhazmi, T. S., Hneiny, L., Baothman, F. A., Alkubeyyer, M., et al. (2023) · Cureus · doi:10.7759/cureus.38373 · Cited by 14
“Bias Assessment As most of the retrieved studies tested the performance of ML models for diagnostic or prognostic accuracy, we assessed the risk of bias using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) [25].”
MethodInfluentialPerformance of a Prediabetes Risk Prediction Model: A Systematic Review
Liu, Y., Feng, W., Lou, J., Qiu, W., Shen, J., Zhu, Z., Hua, Y., Zhang, M., & Billong, L. F. (2022) · Heliyon · doi:10.1016/j.heliyon.2023.e15529 · Cited by 18
Wang, S., Ding, C., Zhang, Q., Hou, M., Chen, Y., Huang, H., Qian, G., Yang, D., Tang, C., Zheng, Y., et al. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2023.1226592 · Cited by 14
“Predictive models have been recommended for clinical risk assessment (4).”
Gao, S., Albu, E., Tuand, K., Cossey, V., Rademakers, F. E., CALSTER, B. V., & Wynants, L. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.07.019 · Cited by 16
Byrne, J. F., Mongan, D., Murphy, J., Healy, C., Föcking, M., Cannon, M. D., & Cotter, D. (2023) · Translational Psychiatry · doi:10.1038/s41398-023-02623-y · Cited by 15
“This systematic review was prospectively registered with PROS-PERO, CRD42022302047.”
MethodRoheger, M., Liebermann‐Jordanidis, H., Krohm, F., Adams, A., & Kalbe, E. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.636355 · Cited by 14
Generative AI and scientific manuscript peer review
Hoyt, R., Limon, A., & Chang, A. C. (2025) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2025.100246 · Cited by 14
Sacoransky, E., Chauhan, Y., & Adams, S. (2025) · American Journal of Roentgenology · doi:10.2214/ajr.25.33759 · Cited by 16
Bias and Oversight in Clinical AI: A Review of Decision Support Tools and Equity Frameworks
Adegunle, F., Chhatwal, K., Arab, S., Alabdaljabar, M. S., Raslan, M. A., Sayed, O., & Goldsweig, A. M. (2026) · Journal of General Internal Medicine · doi:10.1007/s11606-026-10229-5 · Cited by 16
Nguyen, N. H., Patel, S., Gabunilas, J., Qian, A. S., Cecil, A., Jairath, V., Sandborn, W. J., Ohno-Machado, L., Chen, P. L., & Singh, S. (2022) · Clinical and Translational Gastroenterology · doi:10.14309/ctg.0000000000000507 · Cited by 13
Interpretation of meta-analyses
Clephas, P. R. D., & Heesen, M. A. (2022) · Interventional Pain Medicine · doi:10.1016/j.inpm.2022.100120 · Cited by 17
“Other checklists available are the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) [22], Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) [23], Quality In Prognosis Studies (QUIPS) [24], and Prediction model study Risk Of Bias Assessment Tool (PROBAST) [25].”
MethodPsychophysiological models of hypovigilance detection: A scoping review
Marois, A., Kopf, M., Fortin, M., Huot‐Lavoie, M., Martel, A., Boyd, J. G., Gagnon, J., & Archambault, P. M. (2023) · Psychophysiology · doi:10.1111/psyp.14370 · Cited by 15
“…Assessment Tool for Diagnostic Accuracy Studies (QUADAS- 2; Whiting et al., 2011) or the Prediction Model Risk of Bias ASsessment Tool (PROBAST; Wolff et al., 2019) were used to evaluate the quality of each article depending on the type of tool studied: a diagnostic tool vs. a prediction…”
MethodTeixeira, F. J. P., Ahmad, B., Gibatova, V., Ameli, P. A., da Silva, I. R. F., Carneiro, T. S., Roth, W., Ford, J. L., Selfe, T. K., Greer, D. M., et al. (2023) · Critical Care Explorations · doi:10.1097/cce.0000000000000943 · Cited by 13
“The risk of bias will be assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) tool (10).”
Tiberio, P., Gaudio, M., Belloni, S., Pindilli, S., Benvenuti, C., Jacobs, F., Saltalamacchia, G., Zambelli, A., Santoro, A., & Sanctis, R. D. (2023) · Cancers · doi:10.3390/cancers15133424 · Cited by 15
“PY and PN answers are provided to enable judgments when there is an insufficient amount of information with which to answer Y or N confidently [39].”
BackgroundMethodSangariyavanich, E., Ponthongmak, W., Tansawet, A., Theera-Ampornpunt, N., Numthavaj, P., McKay, G. J., Attia, J., & Thakkinstian, A. (2023) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2023.101326 · Cited by 14
“ROB was assessed using the prediction model risk of bias assessment tool (PROBAST) [33], which considers 4 domains.”
MethodDu, Y. H., Guan, C.-J., Li, L. Y., & Gan, P. (2023) · PeerJ · doi:10.7717/peerj.16405 · Cited by 13
“Probast ( Wolff et al., 2019 ) was used to assess the risk of bias in the included original research.”
MethodJia, Y., Cui, N., Jia, T., & Song, J. P. (2024) · ESC Heart Failure · doi:10.1002/ehf2.14696 · Cited by 14
“25 PROBAST comprises four domains, namely participants, predictors, outcome, and analysis.”
BackgroundMethodInfluentialSamadi, M. E., Guzman-Maldonado, J., Nikulina, K., Mirzaieazar, H., Sharafutdinov, K., Fritsch, S. J., & Schuppert, A. (2024) · Scientific Reports · doi:10.1038/s41598-024-55577-6 · Cited by 27
“Different methods have been suggested to account for this issue 33,34 , as well as guidelines like the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis) statement 35 or the PROBAST tool 36 .”
Nikravangolsefid, N., Reddy, S., Truong, H. H., Charkviani, M., Ninan, J., Prokop, L. J., Suppadungsuk, S., Singh, W., KASHANI, K. B., & Garces, J. P. D. (2024) · Journal of Critical Care · doi:10.1016/j.jcrc.2024.154889 · Cited by 15
Prediction models for postpartum stress urinary incontinence: A systematic review
Wang, L., Zhang, M., Sha, K., Qiao, Y., & Dong, Q. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e37988 · Cited by 14
“Predictive model risk of bias was independently assessed by 3 researchers (WLY, ZMH, and QYQ) using the Prediction Model Risk of Bias Assessment Tool (PROBAST) [20].”
MethodJordan, J., Nguyen, C. M., Fletcher, L., & Garbern, S. C. (2024) · Frontiers in Pediatrics · doi:10.3389/fped.2024.1463986 · Cited by 13
Chen, D. C., Huang, S., Papa, N. P., Siva, S., Bolton, D. M., Lawrentschuk, N., Emmett, L., Murphy, D. G., Hofman, M. S., & Perera, M. (2025) · British Journal of Urology · doi:10.1111/bju.16608 · Cited by 15
“For studies assessing SUV max and BCR, the PROBAST was used [51] (Figs S1 and S2).”
Artificial Intelligence-Driven Clinical Decision Support Systems
Alkan, M., Zakariyya, I., Leighton, S., Sivangi, K. B., Anagnostopoulos, C., & Deligianni, F. (2012) · Springer eBooks · doi:10.1007/978-3-032-09253-3_2 · Cited by 12
Prediction models for treatment response in migraine: a systematic review and meta-analysis
Chen, Q., Zhang, J., Cao, B., Hu, Y., Kong, Y., Li, B., & Liu, L. (2025) · The Journal of Headache and Pain · doi:10.1186/s10194-025-01972-x · Cited by 15
“The best model of each study was assessed for risk of bias using Prediction model Risk of Bias Assessment Tool (PROBAST) [28, 29], a tool developed for evaluating the quality of prediction models.”
Kirchgesner, J., Verstockt, B., Adamina, M., Allin, K. H., Allocca, M., Bourgonje, A. R., Burisch, J. M., Doherty, G. A., Dulai, P. S., El‐Hussuna, A., et al. (2025) · Journal of Crohn s and Colitis · doi:10.1093/ecco-jcc/jjaf073 · Cited by 14
Dkeen, N. O. M., Radwan, M. E. D., Zumam, I. A. A., Mohamed, N. A. A. E., Abdelmahmoud, E. M. A., & Magboul, N. M. E. (2025) · Cureus · doi:10.7759/cureus.83961 · Cited by 13
“The risk-of-bias assessment was performed using the Prediction Model Risk-of-Bias Assessment Tool (PROBAST) [10] to systematically evaluate four critical domains: participants (selection bias), predictors (measurement and relevance), outcome (definition and ascertainment), and analysis (statistical…”
Thyroid nodule size and risk of malignancy: a systematic review
Cotter, A., & Jinih, M. (2025) · Discover Oncology · doi:10.1007/s12672-025-02588-y · Cited by 13
Jiang, G., Ma, J., Hao, X., Zhu, Z., Xie, Y., Ji, S., Yan, T., & Luo, L. (2025) · Critical Care · doi:10.1186/s13054-025-05523-3 · Cited by 13
“For prediction studies, we assessed the risk of bias using the Prediction Model Risk of Bias Assessment (PRO-BAST) [11] tool.”
A systematic review of computational modeling of interpersonal dynamics in psychopathology
Zavlis, O., Story, G. W., Friedrich, C., Fonagy, P., & Moutoussis, M. (2025) · Nature Mental Health · doi:10.1038/s44220-025-00465-9 · Cited by 16
“Lack of transparency and openness in methodology Make materials (at least code but also data) openly available 91 Write tutorial papers that detail best research practices 87 Delineate exploratory from confirmatory research 91 Lack of sufficiently similar studies to enable meta-analysis Focus on…”
Bergero, M. Á., Martínez, P., Modina, P., Hosman, R., Villamil, W., Gudiño, R., David, C., & Costa, L. D. (2025) · Scientific Reports · doi:10.1038/s41598-025-16362-1 · Cited by 13
“This study adhered to the TRIPOD-AI guidelines for transparent reporting and the PROBAST-AI tool for assessing risk of bias in diagnostic and prognostic prediction models based on artificial intelligence 16,17 .”
Casal-Guisande, M., Mosteiro, M., Torres‐Durán, M., Comesaña-Campos, A., & Fernández‐Villar, A. (2025) · Expert Review of Respiratory Medicine · doi:10.1080/17476348.2025.2567046 · Cited by 16
“To assess the quality of the models developed in the selected studies, the Prediction model Risk Of Bias ASsessment Tool (PROBAST) [16] was employed.”
Molecular Biomarkers and Machine Learning in Oral Cancer: A Systematic Review and Meta‐Analysis
Ardila, C. M., Vivares‐Builes, A. M., & Pineda‐Vélez, E. (2025) · Oral Diseases · doi:10.1111/odi.70121 · Cited by 13
“The risk of bias of included studies was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST) (Wolff et al. 2019), which evaluates participants, predictors, outcomes, and analysis.”
InfluentialML4H Auditing: From Paper to Practice
Oala, L., Fehr, J., Gilli, L., Balachandran, P., Leite, A. W., Calderón-Ramírez, S., Xie-Li, D., Nobis, G., Alvarado, E. A. M., Jaramillo-Gutierrez, G., et al. (2020) · SERVAL (Université de Lausanne) · Cited by 50
Hussey, A. J., Wing, K., Ferrone, M., & Licskai, C. J. (2021) · International Journal of COPD · doi:10.2147/copd.s338851 · Cited by 13
“Mixed effects multivariable logistic regression models were constructed including all potential prespecified predictor variables with FHT included as a random effect.(42) Potential predictor variables included, identified for their clinical relevance, were age, sex, material deprivation, residential instability, smoki…”
MethodCOVID-19 Pneumonia Classification Based on NeuroWavelet Capsule Network
Monday, H. N., Li, J., Nneji, G. U., Nahar, S., Hossin, M. A., & Jackson, J. K. (2022) · Healthcare · doi:10.3390/healthcare10030422 · Cited by 13
“The overall COVID-19 image-based models perform well; however, some models utilize as few as 10 images, whereas others rely on external validation due to the issues of data availability according to [48].”
Savage, R. S., Messenger, M., Neal, R. D., Ferguson, R., Johnston, C., Lloyd, K., Neal, M., Sansom, N., Selby, P. J., Sharma, N., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-053590 · Cited by 13
Metadata
- Authors
- Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group†
- Affiliations
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Utrecht University (NL)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- 51-58
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907554860
- Cited by
- 2,984
- References
- 42
- Citations by year
- 2026: 921 · 2025: 561 · 2024: 416 · 2023: 343 · 2022: 303 · 2021: 250 · 2020: 140 · 2019: 45
- FWCI
- 83.39
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
- Bristol Research (University of Bristol) (acceptedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
- Radboud Repository (Radboud University) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Apollo (University of Cambridge) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Apollo
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- PROBAST, risk of bias assessment, diagnostic model, prognostic model, applicability, critical appraisal, reporting guidelines, Delphi consensus, systematic reviews, personalized prediction
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Delphi Technique, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.