CoChat
Sign Up
Cited by 95Meta-analysis and systematic reviewsStatistical Methods in EpidemiologyDelphi Technique in Research

The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.

Risk of Bias 2 in Cochrane Reviews: a phased approach for the introduction of new methodology

Open access
Flemyng, Ella, Dwan, Kerry, Moore, Theresa HM, Page, Matthew J, Higgins, Julian PT·2020·Cochrane Database of Systematic Reviews
DOI 10.1002/14651858.ed000148PMC PMC10284096
Open this paper in CoChat — free accountOpen PDF

Abstract

Our confidence in the results of research studies is affected by how they are designed, conducted and reported. In ideal circumstances, all studies would accurately determine the true effect of a health care intervention. However, studies do not take place in an ideal world, and flaws or problems in the way they are conducted could lead to them either overestimating or underestimating the true effect of the intervention. Understanding the potential flaws or problems in studies included in systematic reviews is important as they can distort or bias the results, which means the findings might not accurately represent the truth. Assessing the design, conduct and reporting of studies included in Cochrane Reviews is therefore an important step in understanding whether there is any risk of bias in the results, supporting our confidence in the review's conclusions. Since 2008, authors of Cochrane Reviews have been expected to use the Cochrane risk of bias tool to assess the risk of bias in any randomized trials that they identify, to help users understand the trustworthiness of the review findings. Formal evaluations of user practice over the last decade revealed a need to improve the tool. For example, some bias domains were considered challenging for users (particularly incomplete outcome data and selective outcome reporting), the tool was frequently implemented in non‐standard ways (e.g. with important domains omitted or inappropriate domains added), and the tool was not well suited to cluster randomized trials and crossover trials.[1, 2] To address these limitations and to reflect current understanding of how the causes of bias can influence study results, a new version of the tool, Risk of Bias 2 (RoB 2), was launched in 2019 and included in version 6 of the Cochrane Handbook of Systematic Reviews of Interventions.[3, 4] The revised tool is structured into five domains of bias, according to the stages of a trial in which problems may arise: (1) the randomization process; (2) deviations from intended intervention; (3) missing outcome data; (4) measurement of the outcome; and (5) selection of the reported result. The judgement for each domain is ‘low risk of bias’, ‘some concerns’, or ‘high risk of bias’. In addition, the same three judgement options are available for overall risk of bias. The assessments apply to a specific result from the trial, rather than to the study as a whole. An important innovation is to specify whether the result being assessed is being interpreted as the effect of assigning participants to interventions (the intention‐to‐treat effect) or to the effect of participants adhering to their assigned intervention according to the trial protocol (the per‐protocol effect). The implications of deviations from the intended intervention are different between these, particularly for trials in which participants are aware of their intervention. The implementation of RoB 2 has been a strategic priority for Cochrane and was included in the Strategy to 2020 (www.cochrane.org/about-us/strategy-to-2020) as part of the goal to develop and continue to produce high‐quality, relevant, up‐to‐date systematic reviews. The first Cochrane Review that uses RoB 2 has recently been published.[5] It follows new guidance for reporting RoB 2 assessments in Protocols and Reviews, and illustrates new designs for interactive, results‐level tables for these assessments. The review by Williams and colleagues examined the effects of physical activity interventions for people with congenital heart disease. The RoB 2 tool was used to examine results of the included trials in relation to five outcomes: maximal cardiorespiratory fitness; submaximal cardiorespiratory fitness; health‐related quality of life; physical activity; and muscular strength. The risks of bias were judged by the review team to be similar across these outcomes and in most cases the results were rated overall as having ‘some concerns’. The review team stated that trialists had not reported well the information they needed to judge bias in measurement of the outcome (Domain 4), which includes questions concerning blinding of outcome assessors. Nor had most trials presented sufficient detail in any pre‐specified statistical analysis plans to allow assessment of bias in selection of the reported result (Domain 5). Health‐related quality of life was, however, judged to be at high risk of bias across all randomized trials that reported it: it was self‐reported with a lack of blinding of outcome assessors, and this was thought to have the potential to affect how it was reported. This led to an assessment of ‘very low’ certainty in the health‐related quality of life outcome using GRADE, which was downgraded twice due to risk of bias (also referred to as study limitations). Other outcomes in the review were not downgraded for risk of bias. Introducing new methods and tools for Cochrane Reviews is not without its challenges. Cochrane has 52 Cochrane Review Groups (C

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/14651858.ed000148

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The CONSORT assessment evaluates how an RCT was reported, while the RoB2 assesses the risk of bias of the RCT to help readers understand the reliability of its results.

    “The CONSORT assessment evaluates how an RCT was reported, motivating the description of full reports, whilst the RoB2 assesses the risk of bias of the RCT to help readers understand the reliability of its results (Flemyng et al. 2020).”

    From Adherence to the CONSORT statement of randomized clinical trials on dentin hypersensitivity treatment using mouthwashes (2025) · cites this paper as Flemyng et al. 2020

    The main-text passages describe RoB 2 as assessing risk of bias in randomized trials to help users judge the trustworthiness of findings. They specify that RoB 2 assessments apply to particular trial results, not to the RCT as a whole. The paper does not address CONSORT assessment or its role in describing full reports. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Metadata

Authors
Flemyng, E., Dwan, K., Moore, T. H., Page, M. J., & Higgins, J. P.
Affiliations
  • Cochrane (GB)
  • University of Bristol (GB)
  • Monash University (AU)
ORCID
Venue
Cochrane Database of Systematic Reviews
Publisher
Wiley
Published
2020-11-19
Volume
10
Pages
ED000148-ED000148
ISSN
  • 1361-6137 (ISSN-L)
  • 1465-1858
DOI
10.1002/14651858.ed000148
PMC
PMC10284096
OpenAlex
W3100739476
Cited by
95
References
6
Citations by year
2026: 10 · 2025: 18 · 2024: 34 · 2023: 20 · 2022: 8 · 2021: 3 · 2020: 1
Type
journal article
Language
en
Open access
Locations
  • Cochrane Database of Systematic Reviews (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • Bristol Research (University of Bristol) (submittedVersion, other-oa)
  • Explore Bristol Research (submittedVersion, other-oa)
Funders
National Institute for Health and Care Research (NF-SI-0617-10145)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
phased approach, systematic review methodology
MeSH terms
Systematic Reviews as Topic, Pilot Projects, Quality Control, Bias, Randomized Controlled Trials as Topic, Treatment Outcome

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review