CoChat
Sign Up
Cited by 61Meta-analysis and systematic reviewsStatistical Methods in Clinical TrialsAdvanced Causal Inference Techniques

Treatment effects in randomised trials using routinely collected data for outcome assessment versus traditional trials: meta-research study

Open access
Mc Cord, Kimberly A, Ewald, Hannah, Agarwal, Arnav, Glinz, Dominik, Aghlmandi, Soheila, Ioannidis, John P A, Hemkens, Lars G·2021·BMJ
DOI 10.1136/bmj.n450PMC PMC7926294
Open this paper in CoChat — free account

Abstract

Objective: To compare effect estimates of randomised clinical trials that use routinely collected data (RCD-RCT) for outcome ascertainment with traditional trials not using routinely collected data. Design: Meta-research study. Data source: Studies included in the same meta-analysis in a Cochrane review. Eligibility criteria for study selection: Randomised clinical trials using any type of routinely collected data for outcome ascertainment, including from registries, electronic health records, and administrative databases, that were included in a meta-analysis of a Cochrane review on any clinical question and any health outcome together with traditional trials not using routinely collected data for outcome measurement. Review methods: Effect estimates from trials using or not using routinely collected data were summarised in random effects meta-analyses. Agreement of (summary) treatment effect estimates from trials using routinely collected data and those not using such data was expressed as the ratio of odds ratios. Subgroup analyses explored effects in trials based on different types of routinely collected data. Two investigators independently assessed the quality of each data source. Results: 84 RCD-RCTs and 463 traditional trials on 22 clinical questions were included. Trials using routinely collected data for outcome ascertainment showed 20% less favourable treatment effect estimates than traditional trials (ratio of odds ratios 0.80, 95% confidence interval 0.70 to 0.91, I 2 =14%). Results were similar across various types of outcomes (mortality outcomes: 0.92, 0.74 to 1.15, I 2 =12%; non-mortality outcomes: 0.71, 0.60 to 0.84, I 2 =8%), data sources (electronic health records: 0.81, 0.59 to 1.11, I 2 =28%; registries: 0.86, 0.75 to 0.99, I 2 =20%; administrative data: 0.84, 0.72 to 0.99, I 2 =0%), and data quality (high data quality: 0.82, 0.72 to 0.93, I 2 =0%). Conclusions: Randomised clinical trials using routinely collected data for outcome ascertainment show smaller treatment benefits than traditional trials not using routinely collected data. These differences could have implications for healthcare decision making and the application of real world evidence.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmj.n450

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Routinely collected data, such as from administrative databases, are increasingly used for data collection in randomised trials, although they may be associated with reduced effect estimates.

    “Routinely collected data, such as from administrative databases, are increasingly used for data collection in randomised trials, although they may be associated with reduced effect estimates.389”

    From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [389]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

1 retracted

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Mc Cord, K. A., Ewald, H., Agarwal, A., Glinz, D., Aghlmandi, S., Ioannidis, J. P. A., & Hemkens, L. G.
Affiliations
  • University of Basel (CH)
  • University of Toronto (CA)
  • Stanford Medicine (US)
Corresponding author
Hemkens, Lars G
ORCID
Venue
BMJ
Publisher
BMJ
Published
2021-03-03
Volume
372
Pages
n450
ISSN
1756-1833 (ISSN-L)
DOI
10.1136/bmj.n450
PMC
PMC7926294
OpenAlex
W3135675140
Cited by
61
References
58
Citations by year
2026: 3 · 2025: 12 · 2024: 12 · 2023: 8 · 2022: 12 · 2021: 14
FWCI
6.62
Type
journal article
Language
en
Open access
Licence
Locations
  • BMJ (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • edoc (University of Basel) (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords
administrative data, randomized controlled trials, outcome ascertainment, electronic health records, health registries, treatment effect estimates, real-world evidence, data quality, mortality
MeSH terms
Routinely Collected Health Data, Humans, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review