CoChat

PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies

Open access
Wolff, Robert F., Moons, Karel G.M., Riley, Richard D., Whiting, Penny F., Westwood, Marie, Collins, Gary S., Reitsma, Johannes B., Kleijnen, Jos, Mallett, Sue, for the PROBAST Group† · 2019 · Annals of Internal Medicine

The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.

Abstract

Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.

Overview

Integrity signals

Registry

Crossref · 10.7326/m18-1376

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

The text we found for this paper looks like another document, so its claims and self-check weren’t run. The registry and citation checks still apply.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The text we found for this paper looks like another document, so the self-check wasn’t run.

Citations

Cites

Cited by

See all 2,984

Metadata

Authors
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group†
Affiliations
  • Kleijnen Systematic Reviews (United Kingdom) (GB)
  • Utrecht University (NL)
  • Keele University (GB)
  • University Hospitals Bristol NHS Foundation Trust (GB)
  • Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
  • Maastricht University (NL)
  • Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
Venue
Annals of Internal Medicine
Published
2019-01-01
Volume
170
Issue
1
Pages
51-58
ISSN
  • 0003-4819 (ISSN-L)
  • 1539-3704
OpenAlex
W2907554860
Cited by
2,984
References
42
Citations by year
2026: 921 · 2025: 561 · 2024: 416 · 2023: 343 · 2022: 303 · 2021: 250 · 2020: 140 · 2019: 45
FWCI
83.39
Type
journal article
Language
en
Licence
  • other-oa
  • www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
Locations
  • Annals of Internal Medicine (publishedVersion)
  • PubMed (publishedVersion)
  • Research Publications (Maastricht University) (publishedVersion, other-oa)
  • EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
  • Bristol Research (University of Bristol) (acceptedVersion, other-oa)
  • Keele Research Repository (Keele University) (submittedVersion, cc-by)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
  • Data Archiving and Networked Services (DANS) (submittedVersion)
  • Leiden Repository (Leiden University) (submittedVersion)
  • Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
  • FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
  • University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
  • Radboud Repository (Radboud University) (submittedVersion)
  • Explore Bristol Research (submittedVersion, other-oa)
  • Apollo (University of Cambridge) (submittedVersion)
  • Research Publications (Maastricht University) (submittedVersion)
  • Utrecht University Repository (Utrecht University) (submittedVersion)
  • Apollo
Funders
  • National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
  • National Institute for Health Research School for Primary Care Research (390)
  • National Institute for Health Research Biomedical Research Centre, Oxford
  • National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
PROBAST, risk of bias assessment, diagnostic model, prognostic model, applicability, critical appraisal, reporting guidelines, Delphi consensus, systematic reviews, personalized prediction
MeSH terms
Systematic Reviews as Topic, Decision Support Techniques, Delphi Technique, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.