CoChat
Sign Up
Cited by 150

The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.

Reliability of the revised Cochrane risk-of-bias tool for randomised trials (RoB2) improved with the use of implementation instruction

Minozzi, Silvia, Dwan, Kerry, Borrelli, Francesca, Filippini, Graziella·2022·Journal of Clinical Epidemiology
DOI 10.1016/j.jclinepi.2021.09.021
Open this paper in CoChat — free account

Abstract

Objective to assess the inter-rater reliability (IRR) of the revised Cochrane risk-of-bias tool for randomised trials (RoB2). Methods Four raters independently applied RoB2 on critical and important outcomes of individually randomized parallel-group trials (RCTs) included in the Cochrane Review "Cannabis and cannabinoids for people with multiple sclerosis." We calculated Fleiss' Kappa for multiple raters and time to complete the tool; we performed a calibration exercise on five studies, then we developed an implementation document (ID) specific for the condition, and the intervention addressed by the review with instructions on how to answer the signalling questions of RoB2 tool. We measured IRR before and after the ID adoption RESULTS: Eighty results related to seven outcomes from 16 RCTs were assessed. During calibration exercise we reached no agreement for overall judgment (IRR -0.15); IRR for individual domains ranged from no agreement to fair. Mean time to apply the tool was 168.5 minutes per study. Time to complete the calibration exercise and develop the ID was about 40 hours. After the ID adoption ID, overall agreement increased to slightly (IRR 0.11) for the first five studies and moderate (IRR 0.42) for the remaining 11. IRR for individual domains ranged from no agreement to almost perfect. Mean time to apply the tool decreased to 41 minutes. Conclusion RoB2 tool is comprehensive but complex even for high experienced raters. The development of an ID specific for the…

Overview

Integrity signals

Registry

Crossref · 10.1016/j.jclinepi.2021.09.021

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Flagged first, then by citations

Show all 6 references

Research works printed but not linked: 8

  1. Higgins, 2011, Cochrane bias methods group, cochrane statistical methods group. The cochrane collaboration’s tool for assessing risk of bias in randomised trials, BMJ, 343, d5928, 10.1136/bmj.d5928 · DOI · Paper page
  2. Puljak, 2020, Cochrane risk of bias tool was used inadequately in the majority of non-Cochrane systematic reviews, J Clin Epidemiol, 123, 114, 10.1016/j.jclinepi.2020.03.019 · DOI
  3. Filippini, 2019, Cannabis and cannabinoids for people with multiple sclerosis, Cochrane Database Syst Rev
  4. Landis, 1977, The measurement of observer agreement for categorical data, Biometrics, 33, 159, 10.2307/2529310 · DOI · Paper page
  5. Wright, 2012, Assessment of blinding to treatment allocation in studies of a cannabis-based medicine (Sativex®) in people with multiple sclerosis: a new approach, Trials, 13, 189, 10.1186/1745-6215-13-189 · DOI
  6. Naci, 2019, Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European medicines agency, 2014-16: cross sectional analysis, BMJ, 366, l5221, 10.1136/bmj.l5221 · DOI
  7. de Almeida, 2019, Allocation concealment and intention-to-treat analysis do not influence the treatment effects of physical therapy interventions in low back pain trials: a meta-epidemiologic study, Arch Phys Med Rehabil, 100, 1359, 10.1016/j.apmr.2018.12.036 · DOI
  8. Armijo-Olivo, 2015, What is the influence of randomisation sequence generation and allocation concealment on treatment effects of physical therapy trials? A meta-epidemiological study, BMJ Open, 5, 10.1136/bmjopen-2015-008562 · DOI

Cited by

Sorted by citations

See all 150

Metadata

Authors
Minozzi, S., Dwan, K., Borrelli, F., & Filippini, G.
ORCID
Venue
Journal of Clinical Epidemiology
Publisher
Elsevier BV
Published
2022-01
Volume
141
Pages
99-105
ISSN
0895-4356
DOI
10.1016/j.jclinepi.2021.09.021
Cited by
150
References
6
Language
en
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review