CoChat
Sign Up
Cited by 233Meta-analysis and systematic reviewsStatistical Methods in Clinical TrialsDelphi Technique in Research

The DOI matches its Crossref record; 53 of the 54 references it cites were found in a registry or an index.

Association Between Analytic Strategy and Estimates of Treatment Outcomes in Meta-analyses

Dechartres, Agnes, Altman, Douglas G., Trinquart, Ludovic, Boutron, Isabelle, Ravaud, Philippe·2014·JAMA
DOI 10.1001/jama.2014.8166
Open this paper in CoChat — free account

Abstract

IMPORTANCE: A persistent dilemma when performing meta-analyses is whether all available trials should be included in the meta-analysis. OBJECTIVES: To compare treatment outcomes estimated by meta-analysis of all trials and several alternative analytic strategies: single most precise trial (ie, trial with the narrowest confidence interval), meta-analysis restricted to the 25% largest trials, limit meta-analysis (a meta-analysis model adjusted for small-study effect), and meta-analysis restricted to trials at low overall risk of bias. DATA SOURCES: One hundred sixty-three meta-analyses published between 2008 and 2010 in high-impact-factor journals and between 2011 and 2013 in the Cochrane Database of Systematic Reviews: 92 (705 randomized clinical trials [RCTs]) with subjective outcomes and 71 (535 RCTs) with objective outcomes. DATA SYNTHESIS: For each meta-analysis, the difference in treatment outcomes between meta-analysis of all trials and each alternative strategy, expressed as a ratio of odds ratios (ROR), was assessed considering the dependency between strategies. A difference greater than 30% was considered substantial. RORs were combined by random-effects meta-analysis models to obtain an average difference across the sample. An ROR greater than 1 indicates larger treatment outcomes with meta-analysis of all trials. Subjective and objective outcomes were analyzed separately. RESULTS: Treatment outcomes were larger in the meta-analysis of all trials than in the single most precise trial (combined ROR, 1.13 [95% CI, 1.07-1.19]) for subjective outcomes and 1.03 (95% CI, 1.01-1.05) for objective outcomes). The difference in treatment outcomes between these strategies was substantial in 47 of 92 (51%) meta-analyses of subjective outcomes (meta-analysis of all trials showing larger outcomes in 40/47) and in 28 of 71 (39%) meta-analyses of objective outcomes (meta-analysis of all trials showing larger outcomes in 21/28). The combined ROR for subjective and objective outcomes was, respectively, 1.08 (95% CI, 1.04-1.13) and 1.03 (95% CI, 1.00-1.06) when comparing meta-analysis of all trials and meta-analysis of the 25% largest trials, 1.17 (95% CI, 1.11-1.22) and 1.13 (95% CI, 0.82-1.55) when comparing meta-analysis of all trials and limit meta-analysis, and 0.94 (95% CI, 0.86-1.04) and 1.03 (95% CI, 1.00-1.06) when comparing meta-analysis of all trials and meta-analysis restricted to trials at low risk of bias. CONCLUSIONS AND RELEVANCE: Estimation of treatment outcomes in meta-analyses differs depending on the strategy used. This instability in findings can result in major alterations in the conclusions derived from the analysis and underlines the need for systematic sensitivity analyses.

Overview

Integrity signals

Registry

Crossref · 10.1001/jama.2014.8166

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Flagged first, then by citations

Show all 54 references

1 of the 54 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Dechartres, A., Altman, D. G., Trinquart, L., Boutron, I., & Ravaud, P.
Affiliations
  • Centre de Recherche Epidémiologie et Statistique, INSERM U1153, Paris, France2Centre d’Épidémiologie Clinique, Hôpital Hôtel Dieu, Assistance Publique des Hôpitaux de Paris, Paris, France3Faculté de Médecine, Université Paris Descartes, Sorbonne Paris Cit
  • University of Oxford (GB)
  • Inserm (FR)
ORCID
Venue
JAMA
Publisher
American Medical Association (AMA)
Published
2014-08-13
Volume
312
Issue
6
Pages
623
ISSN
  • 0002-9955 (ISSN-L)
  • 0098-7484
DOI
10.1001/jama.2014.8166
OpenAlex
W2108868101
Cited by
233
References
54
Citations by year
2026: 15 · 2025: 15 · 2024: 25 · 2023: 17 · 2022: 25 · 2021: 15 · 2020: 11 · 2019: 16 · 2018: 18 · 2017: 34 · 2016: 23 · 2015: 11 · 2014: 8
FWCI
24.79
Type
journal article
Language
en
Open access
closed
Locations
  • JAMA (publishedVersion)
  • PubMed (publishedVersion)
Funders
Cancer Research UK (16895)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
analytical strategies, treatment effect estimation, small-study effects, risk of bias, trial size, subjective outcomes, objective outcomes, publication bias, Cochrane Database of Systematic Reviews, randomized controlled trials
MeSH terms
Humans, Research Design, Meta-Analysis as Topic, Confidence Intervals, Odds Ratio, Randomized Controlled Trials as Topic, Treatment Outcome

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review