CoChat

PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration

Open access
Moons, Karel G.M., Wolff, Robert F., Riley, Richard D., Whiting, Penny F., Westwood, Marie, Collins, Gary S., Reitsma, Johannes B., Kleijnen, Jos, Mallett, Sue · 2019 · Annals of Internal Medicine

The DOI matches its Crossref record; 194 of the 201 references it cites were found in a registry or an index.

Abstract

Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.

Overview

Integrity signals

Registry

Crossref · 10.7326/m18-1377

Cites

194 of 194 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

The text we found for this paper looks like another document, so its claims and self-check weren’t run. The registry and citation checks still apply.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The text we found for this paper looks like another document, so the self-check wasn’t run.

Citations

Cites

Research works printed but not linked: 2

  1. 10.1177/0962280218785504.30032705 · doi:10.1177/0962280218785504.30032705
  2. Wong SS, 2003, AMIA Annu Symp Proc, 728

Cited by

See all 1,669

Metadata

Authors
Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S.
Affiliations
  • Utrecht University (NL)
  • Kleijnen Systematic Reviews (United Kingdom) (GB)
  • Keele University (GB)
  • University Hospitals Bristol NHS Foundation Trust (GB)
  • Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
  • Maastricht University (NL)
  • Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
Venue
Annals of Internal Medicine
Published
2019-01-01
Volume
170
Issue
1
Pages
W1-W33
ISSN
  • 0003-4819 (ISSN-L)
  • 1539-3704
OpenAlex
W2907638671
Cited by
1,669
References
201
Citations by year
2026: 359 · 2025: 330 · 2024: 253 · 2023: 236 · 2022: 193 · 2021: 175 · 2020: 96 · 2019: 24
FWCI
56.12
Type
journal article
Language
en
Licence
  • other-oa
  • www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
Locations
  • Annals of Internal Medicine (publishedVersion)
  • PubMed (publishedVersion)
  • Research Publications (Maastricht University) (publishedVersion, other-oa)
  • Utrecht University Repository (Utrecht University) (publishedVersion, other-oa)
  • University of Birmingham Research Portal (University of Birmingham) (publishedVersion)
  • Bristol Research (University of Bristol) (publishedVersion, other-oa)
  • Keele Research Repository (Keele University) (submittedVersion, cc-by)
  • UCL Discovery (University College London) (submittedVersion)
  • University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
  • Explore Bristol Research (submittedVersion, other-oa)
  • Research Publications (Maastricht University) (submittedVersion)
  • Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
Funders
  • National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
  • National Institute for Health Research School for Primary Care Research (390)
  • National Institute for Health Research Biomedical Research Centre, Oxford
  • National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords
PROBAST, risk of bias assessment, applicability, prediction model, diagnostic model, prognostic model, model validation
MeSH terms
Systematic Reviews as Topic, Decision Support Techniques, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.