The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study
Open accessAbstract
OBJECTIVE: To examine whether excluding patients from the analysis of randomised trials are associated with biased estimates of treatment effects and higher heterogeneity between trials. DESIGN: Meta-epidemiological study based on a collection of meta-analyses of randomised trials. DATA SOURCES: 14 meta-analyses including 167 trials that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patient reported pain as an outcome. METHODS: Effect sizes were calculated from differences in means of pain intensity between groups at the end of follow-up, divided by the pooled standard deviation. Trials were combined by using random effects meta-analysis. Estimates of treatment effects were compared between trials with and trials without exclusions from the analysis, and the impact of restricting meta-analyses to trials without exclusions was assessed. RESULTS: 39 trials (23%) had included all patients in the analysis. In 128 trials (77%) some patients were excluded from the analysis. Effect sizes from trials with exclusions tended to be more beneficial than those from trials without exclusions (difference -0.13, 95% confidence interval -0.29 to 0.04). However, estimates of bias between individual meta-analyses varied considerably (tau(2)=0.07). Tests of interaction between exclusions from the analysis and estimates of treatment effects were positive in five meta-analyses. Stratified analyses indicated that differences in effect sizes between trials with and trials without exclusions were more pronounced in meta-analyses with high between trial heterogeneity, in meta-analyses with large estimated treatment benefits, and in meta-analyses of complementary medicine. Restriction of meta-analyses to trials without exclusions resulted in smaller estimated treatment benefits, larger P values, and considerable decreases in between trial heterogeneity. CONCLUSION: Excluding patients from the analysis in randomised trials often results in biased estimates of treatment effects, but the extent and direction of bias is unpredictable. Results from intention to treat analyses should always be described in reports of randomised trials. In systematic reviews, the influence of exclusions from the analysis on estimated treatment effects should routinely be assessed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Effect sizes from trials that excluded participants in their analysis or that used a modified ITT protocol tended to be more beneficial than those from trials without exclusions, demonstrating that the ITT principle is important to preserve the benefits of randomization and keep unbiased estimates when the objective of the trial is to investigate effectiveness.110”
From Effectiveness of Manual Therapy and Therapeutic Exercise for Temporomandibular Disorders: Systematic Review and Meta-Analysis (2016) · cites this paper as [110]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We excluded studies with less than 20 participants at the end of treatment, because of the recognised risk of bias of small numbers (Ioannidis 2005; Nuesch 2009).”
From Psychological therapies for the management of chronic pain (excluding headache) in adults (2020) · cites this paper as Nuesch 2009
Experimental — accuracy not yet measuredIn the citing paper
“exclusion of participants from RCT analyses was shown to distort outcomes42,43”
From The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: A meta-analysis (2018) · cites this paper as [43]
In the citing paper
“An intention-to-treat analysis population is defined by the inclusion of outcome data from all participants in their originally allocated group—this is widely recommended as the preferred analysis population.360 419 420”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [360]
In the citing paper
“Trials at high or unclear risk of bias tend to overestimate benefits and underestimate harms.”
From Should antidepressants be used for major depressive disorder? (2020) · cites this paper as [40]
In the citing paper
“This last criterion of N > 20 at post-treatment assessment is an improvement from the Eccleston 2009a review in which we used an entry of N > 10.We made this change because of the recognised risk of bias of small numbers (Ioannidis 2005;Nuesch 2009); raising the required N further would be desirable but would exclude too many studies.”
From Psychological therapies for the management of chronic pain (excluding headache) in adults (2012) · cites this paper as Nuesch 2009
In the citing paper
“the overall tendency for poorer quality trials to produce more positive results (Furlan 2001;Ioannidis 2005;Nuesch 2009).”
From Psychological therapies for the management of chronic pain (excluding headache) in adults (2012) · cites this paper as Nuesch 2009
In the citing paper
“given the demonstrated association between small numbers and bias (Ioannidis 2005; Moore 2010; Nuesch 2009).”
From Psychological therapies for the management of chronic pain (excluding headache) in adults (2012) · cites this paper as Nuesch 2009
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,603
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835 · cited 1×
“In our study we did not rely on statements in the trial reports on whether an intention to treat analysis was done or not. Rather we required explicit information about the flow of patients through the study”
MethodSelf-citationAssessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309 · cited 1×
“In clinical trials, deviations from protocol and losses to follow-up often lead to the exclusion of some randomised patients from the analysis.”
BackgroundSelf-citationMoher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
Self-citationWood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538
Self-citationMeta-analyses involving cross-over trials: methodological issues
Elbourne, D. R., Altman, D. G., Higgins, J. P., Curtin, F., Worthington, H. V., & Vail, A. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.140 · Cited by 2,305
Self-citationGluud, L. L., Villumsen, J., & Gluud, C. N. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-11-200112040-00010 · Cited by 2,266
What is meant by intention to treat analysis? Survey of published randomised controlled trials
Hollis, S., & Campbell, F. (1999) · BMJ · doi:10.1136/bmj.319.7211.670 · Cited by 2,160
Bayesian Approaches to Clinical Trials and Health‐Care Evaluation
Spiegelhalter, D. J., Abrams, K. R., & Myles, J. P. (2003) · Wiley eBooks · doi:10.1002/0470092602 · Cited by 1,687
Moher, D., Schulz, K. F., Altman, D. G., & for the CONSORT Group* (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00011 · Cited by 1,426
Self-citationMeta-analysis: chronic disease self-management programs for older adults
Chodosh, J., Morton, S. C., Mojica, W., Maglione, M., Suttorp, M. J., Hilton, L., Rhodes, S., & Shekelle, P. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-143-6-200509200-00007 · Cited by 790
Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis
Fergusson, D., Aaron, S. D., Guyatt, G., & Hébert, P. (2002) · BMJ · doi:10.1136/bmj.325.7365.652 · Cited by 708
Glucosamine therapy for treating osteoarthritis
Towheed, T., Maxwell, L. J., Anastassiades, T., Shea, B., Houpt, J., Welch, V. A., Hochberg, M. C., & Wells, G. A. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002946.pub2 · Cited by 688
Christensen, R., Bartels, E. M., Astrup, A. V., & Bliddal, H. (2007) · Annals of the Rheumatic Diseases · doi:10.1136/ard.2006.065904 · Cited by 660
The Coronary Drug Project Research Group (1980) · New England Journal of Medicine · doi:10.1056/nejm198010303031804 · Cited by 659
Melander, H., Ahlqvist-Rastad, J., Meijer, G., & Beermann, B. (2003) · BMJ · doi:10.1136/bmj.326.7400.1171 · Cited by 634
Intra-articular Hyaluronic Acid in Treatment of Knee Osteoarthritis
Lo, G. H., LaValley, M. P., McAlindon, T. E., & Felson, D. T. (2003) · JAMA · doi:10.1001/jama.290.23.3115 · Cited by 599
Acetaminophen for osteoarthritis
Towheed, T., Maxwell, L. J., Judd, M., Catton, M., Hochberg, M. C., & Wells, G. A. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004257.pub2 · Cited by 568
Balk, E. M. (2002) · JAMA · doi:10.1001/jama.287.22.2973 · Cited by 478
Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials
Pildal, J., Hrobjartsson, A., Jorgensen, K., Hilden, J., Altman, D., & Gotzsche, P. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym087 · Cited by 472
Self-citationOptimal search strategies for retrieving systematic reviews from Medline: analytical survey
Montori, V. M., Wilczynski, N. L., Morgan, D., & Haynes, R. B. (2005) · BMJ · doi:10.1136/bmj.38336.804167.47 · Cited by 422
Controversy in Counting and Attributing Events in Clinical Trials
Sackett, D. L., & Gent, M. (1979) · New England Journal of Medicine · doi:10.1056/nejm197912273012602 · Cited by 413 · cited 1×
“Patients excluded after randomisation are unlikely to be representative of patients remaining in the trial. For example, patients may not be available for follow-up because they have an acute exacerbation of their condition or severe side effects,”
BackgroundQuality of Reporting of Randomized Trials as a Measure of Methodologic Quality
Huwiler-Müntener, K. (2002) · JAMA · doi:10.1001/jama.287.21.2801 · Cited by 397 · cited 1×
“our study is based on published information and depends on the quality of the trial reports. Even though the quality of reporting is generally low,”
MethodSterne, J. A. C., Jüni, P., Schulz, K. F., Altman, D. G., Bartlett, C., & Egger, M. (2002) · Statistics in Medicine · doi:10.1002/sim.1184 · Cited by 382
Self-citationBjordal, J. M., Johnson, M. I., Lopes-Martins, R. A., Bogen, B. E., Chow, R. T., & Ljunggren, A. E. (2007) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-8-51 · Cited by 329
Value of Flow Diagrams in Reports of Randomized Controlled Trials
Egger, M. (2001) · JAMA · doi:10.1001/jama.285.15.1996 · Cited by 317 · cited 1×
“In our study we did not rely on statements in the trial reports on whether an intention to treat analysis was done or not. Rather we required explicit information about the flow of patients through the study”
MethodBayesian Approaches to Clinical Trials and Health-Care Evaluation
Best, M. A. (2004) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2004.00347_17.x · Cited by 314
Meta-analysis: Chondroitin for Osteoarthritis of the Knee or Hip
Reichenbach, S., Sterchi, R., Scherer, M., Trelle, S., Bürgi, E., Bürgi, U., Dieppe, P. A., & Jüni, P. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-8-200704170-00009 · Cited by 314 · cited 1×
“The primary outcome was patient reported pain. If different pain outcomes were reported we extracted one outcome per study according to a hierarchy described previously.”
MethodSelf-citationSoares, H. P., Daniels, S., Kumar, A., Clarke, M. J., Scott, C., Swann, S., & Djulbegović, B. (2004) · BMJ · doi:10.1136/bmj.328.7430.22 · Cited by 310
Bjordal, J. M., Klovning, A., Ljunggren, A. E., & Slørdal, L. (2006) · European Journal of Pain · doi:10.1016/j.ejpain.2006.02.013 · Cited by 297
Meta-analysis: Acupuncture for Osteoarthritis of the Knee
Manheimer, E. W., Linde, K., Lao, L., Bouter, L. M., & Berman, B. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-12-200706190-00008 · Cited by 250
Osteoarthritis: rational approach to treating the individual
Jüni, P., Reichenbach, S., & Dieppe, P. A. (2006) · Best Practice & Research Clinical Rheumatology · doi:10.1016/j.berh.2006.05.002 · Cited by 237
Self-citationInvestigating patient exclusion bias in meta-analysis
Tierney, J. F. (2004) · International Journal of Epidemiology · doi:10.1093/ije/dyh300 · Cited by 221 · cited 1×
“In clinical trials, deviations from protocol and losses to follow-up often lead to the exclusion of some randomised patients from the analysis.”
BackgroundWei, Q., Wu, J. T., MacDonald, R., Monga, M., & Wilt, T. J. (2002) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004286 · Cited by 220
The Case of the Missing Data: Methods of Dealing with Dropouts and other Research Vagaries
Streiner, D. L. (2002) · The Canadian Journal of Psychiatry · doi:10.1177/070674370204700111 · Cited by 215 · cited 1×
“but leads to overly precise estimates and potential bias.”
BackgroundEfficacy and safety of opioids for osteoarthritis: a meta-analysis of randomized controlled trials
Avouac, J., Gossec, L., & Dougados, M. (2007) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2007.02.006 · Cited by 165
Meta-analyses involving cross-over trials: methodological issues
Stedman, M. R., Curtin, F., Elbourne, D. R., Kesselheim, A. S., & Brookhart, M. A. (2009) · International Journal of Epidemiology · doi:10.1093/ije/dyp345 · Cited by 148 · cited 1×
“Two reviewers used a standardised form to independently extract data from the original reports of individual trials on interventions, funding, year of publication, publication language, design, study size, blinding of patients, losses to follow-up, exclusions, handling of missing data, and results. When necessary we a…”
MethodGravel, J., Opatrny, L., & Shapiro, S. (2007) · Clinical Trials · doi:10.1177/1740774507081223 · Cited by 138 · cited 1×
“When restricting meta-analyses to trials without exclusions, P values increased in most cases and six meta-analyses lost statistical significance at P=0.05 (fig 4). This increase in P values was not only due to a loss of statistical power.”
MethodInfluentialExercise for osteoarthritis of the hip or knee
Fransen, M., McConnell, S., & Bell, M. (2003) · Physiotherapy · doi:10.1016/s0031-9406(05)60174-5 · Cited by 135
A Meta-analysis of Controlled Clinical Studies With Diacerein in the Treatment of Osteoarthritis
Rintelen, B., Neumann, K., & Leeb, B. F. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.17.1899 · Cited by 122
Gluud, L. L., Thorlund, K., Gluud, C. N., Woods, L., Harris, R., & Sterne, J. A. C. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-3-200808050-00023 · Cited by 85
Baron, G., Boutron, I., Giraudeau, B., & Ravaud, P. (2005) · Arthritis & Rheumatism · doi:10.1002/art.21116 · Cited by 59 · cited 1×
“Trials without exclusions more often reported imputations of missing data than those with exclusions. The last observation carried forward approach was used most often: missing values were replaced by the last value observed. This method is popular for imputation of missing data in musculoskeletal research”
MethodCommentary: Empirical evidence of attrition bias in clinical trials
Jüni, P. (2004) · International Journal of Epidemiology · doi:10.1093/ije/dyh406 · Cited by 49
Baron, G., Ravaud, P., Samson, A., & Giraudeau, B. (2007) · Arthritis Care & Research · doi:10.1002/art.23253 · Cited by 38 · cited 1×
“Multiple imputation is more difficult to carry out but avoids those problems”
Background8 The Case of the Missing Data: Methods of Dealing with Dropouts and Other Research Vagaries
Streiner, D. L. (2012) · University of Toronto Press eBooks · doi:10.3138/9781442662827-010 · Cited by 23
Gluud, L. L., Thorlund, K., Gluud, C. N., Woods, L., Harris, R., & Sterne, J. A. C. (2008) · Explore Bristol Research · Cited by 15
Issues with using baseline in last observation carried forward analysis
Julious, S. A., & Mullee, M. (2007) · Pharmaceutical Statistics · doi:10.1002/pst.311 · Cited by 13
Statistical methods inArthritis & Rheumatism: Current trends
Kim, M. (2006) · Arthritis & Rheumatism · doi:10.1002/art.22251 · Cited by 10
Gluud, C. N., Wood, A. J. G., Jüni, P., Wood, L., Altman, D. G., Schulz, K. F., Gluud, L. L., Martin, R. M., Egger, M., & Sterne, J. A. C. (2020) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/p8aq-rs50 · Cited by 4
Self-citation
Cited by
Sorted by citations
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251
Self-citationPsychological therapies for the management of chronic pain (excluding headache) in adults
Williams, A. C. D. C., Eccleston, C., & Morley, S. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007407.pub3 · Cited by 1,634
Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049
“There is recent empirical evidence of bias in the results of randomized, controlled trials due to attrition (26, 27) and selective outcome reporting (28, 29).”
BackgroundSelf-citationMeta‐Analysis in Medical Research
Haidich, A. (2005) · Wiley eBooks · doi:10.1002/9780470994894 · Cited by 692
Savović, J., Jones, H., Altman, D., Harris, R., Jűni, P., Pildal, J., Als-Nielsen, B., Balk, E., Gluud, C., Gluud, L., et al. (2012) · Health Technology Assessment · doi:10.3310/hta16350 · Cited by 668
“Blinding All 71 (36) 725 290 (40) 33 (17) 286 84 (29) Mortality 13 (5) 119 39 (33) 5 (3) 48 10 (21) Objective 20 (12) 208 81 (39) 9 (5) 63 23 (37) Subjective 38 (19) 398 170 (43) 19 (9) 175 51 (29)”
Self-citation
Metadata
- Authors
- Nuesch, E., Trelle, S., Reichenbach, S., Rutjes, A. W. S., Burgi, E., Scherer, M., Altman, D. G., & Juni, P.
- Affiliations
- University of Bern (CH)
- University of Göttingen (DE)
- University of Oxford (GB)
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2009-09-07
- Volume
- 339
- Issue
- sep07 1
- Pages
- b3244-b3244
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- PMC
- PMC2739282
- OpenAlex
- W2057810715
- Cited by
- 248
- References
- 52
- Citations by year
- 2026: 9 · 2025: 11 · 2024: 8 · 2023: 9 · 2022: 7 · 2021: 15 · 2020: 20 · 2019: 11 · 2018: 11 · 2017: 15 · 2016: 15 · 2015: 17 · 2014: 17 · 2013: 20 · 2012: 22
- FWCI
- 1.55
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by-nc)
- Bern Open Repository and Information System (University of Bern) (submittedVersion, cc-by-nc)
- GoeScholar The Publication Server of the Georg-August-Universität Göttingen (Georg-August-Universität Göttingen) (submittedVersion, cc-by)
- Open Access CRIS of the University of Bern
- Funders
- National Science Foundation
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (115067, 104762, 66377, 066377, 66378)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- randomized controlled trials, bias in treatment effect estimates, between-trial heterogeneity, intention-to-treat analysis, osteoarthritis, hip osteoarthritis, knee osteoarthritis, patient-reported pain, complementary therapy, meta-epidemiology
- MeSH terms
- Data Collection, Humans, Pain, Meta-Analysis as Topic, Osteoarthritis, Hip, Bias, Randomized Controlled Trials as Topic, Sample Size, Osteoarthritis, Knee
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
