CoChat
Sign Up
Cited by 248Meta-analysis and systematic reviewsStatistical Methods in Clinical TrialsPharmaceutical industry and healthcare

The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study

Open access
Nuesch, E., Trelle, S., Reichenbach, S., Rutjes, A. W S, Burgi, E., Scherer, M., Altman, D. G, Juni, P.·2009·BMJ
DOI 10.1136/bmj.b3244PMC PMC2739282
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVE: To examine whether excluding patients from the analysis of randomised trials are associated with biased estimates of treatment effects and higher heterogeneity between trials. DESIGN: Meta-epidemiological study based on a collection of meta-analyses of randomised trials. DATA SOURCES: 14 meta-analyses including 167 trials that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patient reported pain as an outcome. METHODS: Effect sizes were calculated from differences in means of pain intensity between groups at the end of follow-up, divided by the pooled standard deviation. Trials were combined by using random effects meta-analysis. Estimates of treatment effects were compared between trials with and trials without exclusions from the analysis, and the impact of restricting meta-analyses to trials without exclusions was assessed. RESULTS: 39 trials (23%) had included all patients in the analysis. In 128 trials (77%) some patients were excluded from the analysis. Effect sizes from trials with exclusions tended to be more beneficial than those from trials without exclusions (difference -0.13, 95% confidence interval -0.29 to 0.04). However, estimates of bias between individual meta-analyses varied considerably (tau(2)=0.07). Tests of interaction between exclusions from the analysis and estimates of treatment effects were positive in five meta-analyses. Stratified analyses indicated that differences in effect sizes between trials with and trials without exclusions were more pronounced in meta-analyses with high between trial heterogeneity, in meta-analyses with large estimated treatment benefits, and in meta-analyses of complementary medicine. Restriction of meta-analyses to trials without exclusions resulted in smaller estimated treatment benefits, larger P values, and considerable decreases in between trial heterogeneity. CONCLUSION: Excluding patients from the analysis in randomised trials often results in biased estimates of treatment effects, but the extent and direction of bias is unpredictable. Results from intention to treat analyses should always be described in reports of randomised trials. In systematic reviews, the influence of exclusions from the analysis on estimated treatment effects should routinely be assessed.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmj.b3244

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Effect sizes from trials that excluded participants in their analysis or that used a modified ITT protocol tended to be more beneficial than those from trials without exclusions, demonstrating that the ITT principle is important to preserve the benefits of randomization and keep unbiased estimates when the objective of the trial is to investigate effectiveness.

    “Effect sizes from trials that excluded participants in their analysis or that used a modified ITT protocol tended to be more beneficial than those from trials without exclusions, demonstrating that the ITT principle is important to preserve the benefits of randomization and keep unbiased estimates when the objective of the trial is to investigate effectiveness.110”

    From Effectiveness of Manual Therapy and Therapeutic Exercise for Temporomandibular Disorders: Systematic Review and Meta-Analysis (2016) · cites this paper as [110]

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • The authors excluded studies with less than 20 participants at the end of treatment, because of the recognised risk of bias of small numbers.

    “We excluded studies with less than 20 participants at the end of treatment, because of the recognised risk of bias of small numbers (Ioannidis 2005; Nuesch 2009).”

    From Psychological therapies for the management of chronic pain (excluding headache) in adults (2020) · cites this paper as Nuesch 2009

    The Discussion and Conclusions say that excluding randomised patients from an analysis can bias treatment-effect estimates. They do not address bias from studies having fewer than 20 participants at the end of treatment. Searches for a 20-participant threshold and small-sample bias found no supporting passage. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

  • Exclusion of participants from RCT analyses was shown to distort outcomes.

    “exclusion of participants from RCT analyses was shown to distort outcomes42,43”

    From The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: A meta-analysis (2018) · cites this paper as [43]

  • An intention-to-treat analysis population is widely recommended as the preferred analysis population.

    “An intention-to-treat analysis population is defined by the inclusion of outcome data from all participants in their originally allocated group—this is widely recommended as the preferred analysis population.360 419 420”

    From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [360]

  • Trials at high or unclear risk of bias tend to overestimate benefits and underestimate harms.

    “Trials at high or unclear risk of bias tend to overestimate benefits and underestimate harms.”

    From Should antidepressants be used for major depressive disorder? (2020) · cites this paper as [40]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Nuesch, E., Trelle, S., Reichenbach, S., Rutjes, A. W. S., Burgi, E., Scherer, M., Altman, D. G., & Juni, P.
Affiliations
  • University of Bern (CH)
  • University of Göttingen (DE)
  • University of Oxford (GB)
ORCID
Venue
BMJ
Publisher
BMJ
Published
2009-09-07
Volume
339
Issue
sep07 1
Pages
b3244-b3244
ISSN
  • 0007-1447 (ISSN-L)
  • 0959-8138
  • 1468-5833
DOI
10.1136/bmj.b3244
PMC
PMC2739282
OpenAlex
W2057810715
Cited by
248
References
52
Citations by year
2026: 9 · 2025: 11 · 2024: 8 · 2023: 9 · 2022: 7 · 2021: 15 · 2020: 20 · 2019: 11 · 2018: 11 · 2017: 15 · 2016: 15 · 2015: 17 · 2014: 17 · 2013: 20 · 2012: 22
FWCI
1.55
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • BMJ (publishedVersion, cc-by-nc)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
  • IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by-nc)
  • Bern Open Repository and Information System (University of Bern) (submittedVersion, cc-by-nc)
  • GoeScholar The Publication Server of the Georg-August-Universität Göttingen (Georg-August-Universität Göttingen) (submittedVersion, cc-by)
  • Open Access CRIS of the University of Bern
Funders
  • National Science Foundation
  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (115067, 104762, 66377, 066377, 66378)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Pharmacology › Pharmaceutical industry and healthcare
Keywords
randomized controlled trials, bias in treatment effect estimates, between-trial heterogeneity, intention-to-treat analysis, osteoarthritis, hip osteoarthritis, knee osteoarthritis, patient-reported pain, complementary therapy, meta-epidemiology
MeSH terms
Data Collection, Humans, Pain, Meta-Analysis as Topic, Osteoarthritis, Hip, Bias, Randomized Controlled Trials as Topic, Sample Size, Osteoarthritis, Knee

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review