Cited by 274Meta-analysis and systematic reviewsDelphi Technique in ResearchStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index; 2 of the 21 sources its claims cite were checked for support.

The Cochrane risk of bias assessment tool 2 (RoB 2) versus the original RoB: A perspective on the pros and cons

Open access

Abstract

Background and Aims: Critical appraisal or risk of bias assessment is a fundamental part of systematic reviews that clarifies the degree to which included research articles are qualified and reliable. Version 2 of the Cochrane tool for assessing the risk of bias in randomized trials (RoB 2), the updated version of the first tool, was released in 2019. Here, we have compared these two versions of Cochrane risk of bias assessment tools and highlighted the pros and cons of RoB 2. Methods: Statistical analysis and methodology is not applicable to this article as no new data were created or analyzed in this study. Results: The overall approach in RoB 2 is that by answering some signaling questions after the specification of results, effects of interest, and sources of information, an overall judgment for the quality of each study is reached. Accordingly, in the original version of the Cochrane RoB tool, the judgment can be in three different conclusions, including low, unclear, and high risk of bias. The most prominent difference in bias domains is the removal of “other bias” domain being replaced by “overall bias” judgment. Also, the most common presentation types of Cochrane risk of bias assessments are the “summary” and “graph” which are generated by Review Manager, web‐based applications, or packages in R software. Conclusion: The RoB 2 tool, compared to the original RoB, has improved and is the recommended version by the Cochrane Collaboration for quality assessment of randomized controlled trials. It is recommended to consider funding source, duration of follow‐up, declaration of data availability, the status of baseline characteristics between groups, and sample size calculation methods in further revisions of the Cochrane risk of bias assessment tools.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This perspective article compares the Cochrane risk of bias tool version 2 (RoB 2, released 2019) with the original RoB 1 for randomized trials, arguing RoB 2 is improved and Cochrane-recommended but more complex and time-consuming. The authors recommend that future revisions consider funding source, follow-up duration, data availability, baseline characteristics, and sample size calculation methods.

Key findings

  • RoB 2 assesses five bias domains (randomization process, deviations from intended interventions, missing outcome data, measurement of outcomes, selection of the reported result) plus an "overall bias" judgment, whereas RoB 1 had six domains including "other bias"; the removal of "other bias" is described as the most prominent domain difference and as improving clarity.
  • RoB 2 uses structured signaling questions answered at five levels (yes, probably yes, no, probably no, no information) leading to judgments of low risk, some concerns, or high risk, replacing RoB 1's low/unclear/high judgments; RoB 2 focuses on results rather than outcomes.
  • RoB 2 is more resource-intensive: a systematic review of 113 studies reported an average of 358 min per article, versus 10–60 min per article for the original RoB tool; two studies found no significant inter-rater reliability (IRR: −0.15) and slightly higher IRR (IRR: 0.16) for overall judgment, while use of RoB 2 was significantly associated with improved methodological quality of systematic reviews (p = 0.007).
  • Drawing on Babic et al.'s survey of 768 Cochrane reviews including 11,369 RCTs, 87% of reviews reported the "other bias" domain with 5762 explanations categorized into 31 groups; the five most frequent categories were baseline characteristics (21.4%), funding (15.6%), sample size (8.1%), reporting (7.6%), and conflicts of interest (5.8%). A Cochrane review of 75 articles found industry-funded studies more often had favorable efficacy results (RR: 1.27; 95% CI: 1.17–1.37), suggesting an "industry bias" not assessed by RoB 2.

Methods

  • Perspective article; the authors state that statistical analysis and methodology are not applicable because no new data were created or analyzed.
  • The comparison focuses on RoB 2 for parallel study design versus RoB 1, drawing on previously published studies and the Cochrane Handbook to support the discussion.
  • No primary data were collected; results from prior studies are cited as supporting evidence, and suggestions for future tools are based on previous findings and the authors' opinions.

Limitations

  • Stated by the authors: it is a perspective article that only critiques the pros and cons of RoB 2 versus RoB 1 and offers factors for future iterations; it did not collect or report primary original data; it assessed only Cochrane RoB 2 with a focus on the parallel study design version and did not evaluate other quality assessment tools for clinical trials or observational studies; the suggestions are based on previous findings and authors' opinions, and further research, pilot studies, and expert consensus may be needed to evaluate their implementation.
  • Evident from the text: RoB 2 lacks supplements for some trial types (e.g., trials without a control arm, factorial-design RCTs), and some domains remain subjective, with signaling-question answers (e.g., on deviations from intended intervention) often being "no information" or "probably no" due to unavailable protocols, potentially leading to discrepant judgments.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1002/hsr2.2165

Cites

17 of 17 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Summary of risk of bias assessment for four samples included studies using Cochrane risk of bias assessments tool version 1 (A) and version 2 (RoB 2) (B). “+”: low risk of bias; “?”: unclear risk of bias; “−”: high risk of bias for version 1.
Figure 1 Summary of risk of bias assessment for four samples included studies using Cochrane risk of bias assessments tool version 1 (A) and version 2 (RoB 2) (B). “+”: low risk of bias; “?”: unclear risk of bias; “−”: high risk of bias for version 1.CC BY · PubMed Central

Verification

Claims the paper makes

16 Not checked yet · 14 Own interpretation · 2 Supported · 1 Sources not found · 1 No basis given

Verify all 16 references — free account

Show all 10 claims

  • It may state two separate claims as one. 11
  • Probably not a substantial claim. 15
24 of 24
  • The paper relies on this without support.

    The implications of the results for researchers and methodologists and how the changes can lead to high-quality systematic reviews were not mentioned in previous publications.

    “However, the implications of the results for researchers and methodologists and how the changes can lead to the production of high‐quality systematic reviews were not mentioned.”
  • The paper relies on this without support.

    Some domains that were omitted from the original version of RoB assessment were not analyzed in depth in previous publications.

    “In addition, some domains that were omitted from the original version of RoB assessment were not analyzed in depth in the previous publications.”
  • The paper relies on this without support.

    The RoB 2 includes supplements for parallel, cluster-randomized, and cross-over designs of RCTs, whereas RoB 1 has one guidance and iteration.

    “Despite the RoB 1 that has one guidance and iteration, the RoB 2 includes other supplements for parallel, cluster‐randomized, and cross‐over designs of RCTs.”
  • The paper relies on this without support.

    Both versions of the Cochrane RoB assessment tools use an option for RoB that cannot be categorized into low or high RoB; the original uses 'unclear' and RoB 2 uses 'some concerns'.

    “Both versions of the Cochrane RoB assessment tools use another option for RoB that cannot be categorized into low or high RoB. In this regard, the original version uses “unclear” and the RoB 2 uses “some concerns.””
  • The paper relies on this without support.

    The RoB 1 and RoB 2 include the 'reporting bias' domain and 'bias in the selection of the reported result,' respectively, to assess reporting bias.

    “The RoB 1 and RoB 2 include the “reporting bias” domain and “bias in the selection of the reported result,” respectively, to assess this outcome.”
  • The paper relies on this without support.

    The most common presentation types of Cochrane risk of bias assessments are the 'summary' and 'graph' generated by Review Manager, web-based applications, or R packages.

    “The most common presentation types of Cochrane risk of bias assessments are the “summary” and “graph” which are generated by Review Manager (RevMan) (Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014) or RevMan Web application.”
6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 17 references

Research works printed but not linked: 1

  1. Higgins JPT, 2021, Cochrane Handbook for Systematic Reviews of Interventions Version 6.2 (Updated February 2021)

Cited by

Sorted by citations

Metadata

Authors
Nejadghaderi, S. A., Balibegloo, M., & Rezaei, N.
Affiliations
  • Kerman University of Medical Sciences (IR)
  • Universal Scientific Education and Research Network (IR)
Corresponding author
Rezaei, Nima
Venue
Health Science Reports
Publisher
Wiley
Published
2024-06
Volume
7
Issue
6
Pages
e2165-e2165
ISSN
2398-8835 (ISSN-L)
OpenAlex
W4399330391
Cited by
274
References
17
Citations by year
2026: 177 · 2025: 89 · 2024: 8
FWCI
33.33
Type
journal article
Language
en
Open access
Locations
  • Health Science Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords
risk of bias assessment, critical appraisal, randomized controlled trials, trial quality assessment, systematic review methodology, funding disclosure, sample size determination

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review