The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study
Open accessAbstract
OBJECTIVE: To examine whether excluding patients from the analysis of randomised trials are associated with biased estimates of treatment effects and higher heterogeneity between trials. DESIGN: Meta-epidemiological study based on a collection of meta-analyses of randomised trials. DATA SOURCES: 14 meta-analyses including 167 trials that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patient reported pain as an outcome. METHODS: Effect sizes were calculated from differences in means of pain intensity between groups at the end of follow-up, divided by the pooled standard deviation. Trials were combined by using random effects meta-analysis. Estimates of treatment effects were compared between trials with and trials without exclusions from the analysis, and the impact of restricting meta-analyses to trials without exclusions was assessed. RESULTS: 39 trials (23%) had included all patients in the analysis. In 128 trials (77%) some patients were excluded from the analysis. Effect sizes from trials with exclusions tended to be more beneficial than those from trials without exclusions (difference -0.13, 95% confidence interval -0.29 to 0.04). However, estimates of bias between individual meta-analyses varied considerably (tau(2)=0.07). Tests of interaction between exclusions from the analysis and estimates of treatment effects were positive in five meta-analyses. Stratified analyses indicated that differences in effect sizes between trials with and trials without exclusions were more pronounced in meta-analyses with high between trial heterogeneity, in meta-analyses with large estimated treatment benefits, and in meta-analyses of complementary medicine. Restriction of meta-analyses to trials without exclusions resulted in smaller estimated treatment benefits, larger P values, and considerable decreases in between trial heterogeneity. CONCLUSION: Excluding patients from the analysis in randomised trials often results in biased estimates of treatment effects, but the extent and direction of bias is unpredictable. Results from intention to treat analyses should always be described in reports of randomised trials. In systematic reviews, the influence of exclusions from the analysis on estimated treatment effects should routinely be assessed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Effect sizes from trials that excluded participants in their analysis or that used a modified ITT protocol tended to be more beneficial than those from trials without exclusions, demonstrating that the ITT principle is important to preserve the benefits of randomization and keep unbiased estimates when the objective of the trial is to investigate effectiveness.110”
From Effectiveness of Manual Therapy and Therapeutic Exercise for Temporomandibular Disorders: Systematic Review and Meta-Analysis (2016) · cites this paper as [110]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We excluded studies with less than 20 participants at the end of treatment, because of the recognised risk of bias of small numbers (Ioannidis 2005; Nuesch 2009).”
From Psychological therapies for the management of chronic pain (excluding headache) in adults (2020) · cites this paper as Nuesch 2009
Experimental — accuracy not yet measuredIn the citing paper
“exclusion of participants from RCT analyses was shown to distort outcomes42,43”
From The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: A meta-analysis (2018) · cites this paper as [43]
In the citing paper
“An intention-to-treat analysis population is defined by the inclusion of outcome data from all participants in their originally allocated group—this is widely recommended as the preferred analysis population.360 419 420”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [360]
In the citing paper
“Trials at high or unclear risk of bias tend to overestimate benefits and underestimate harms.”
From Should antidepressants be used for major depressive disorder? (2020) · cites this paper as [40]
In the citing paper
“This last criterion of N > 20 at post-treatment assessment is an improvement from the Eccleston 2009a review in which we used an entry of N > 10.We made this change because of the recognised risk of bias of small numbers (Ioannidis 2005;Nuesch 2009); raising the required N further would be desirable but would exclude too many studies.”
From Psychological therapies for the management of chronic pain (excluding headache) in adults (2012) · cites this paper as Nuesch 2009
In the citing paper
“the overall tendency for poorer quality trials to produce more positive results (Furlan 2001;Ioannidis 2005;Nuesch 2009).”
From Psychological therapies for the management of chronic pain (excluding headache) in adults (2012) · cites this paper as Nuesch 2009
In the citing paper
“given the demonstrated association between small numbers and bias (Ioannidis 2005; Moore 2010; Nuesch 2009).”
From Psychological therapies for the management of chronic pain (excluding headache) in adults (2012) · cites this paper as Nuesch 2009
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,603
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835 · cited 1×
“In our study we did not rely on statements in the trial reports on whether an intention to treat analysis was done or not. Rather we required explicit information about the flow of patients through the study”
MethodSelf-citationAssessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309 · cited 1×
“In clinical trials, deviations from protocol and losses to follow-up often lead to the exclusion of some randomised patients from the analysis.”
BackgroundSelf-citationMoher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
Self-citationWood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538
Self-citationMeta-analyses involving cross-over trials: methodological issues
Elbourne, D. R., Altman, D. G., Higgins, J. P., Curtin, F., Worthington, H. V., & Vail, A. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.140 · Cited by 2,305
Self-citationGluud, L. L., Villumsen, J., & Gluud, C. N. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-11-200112040-00010 · Cited by 2,266
Flagged first, then by citations
Cited by
Sorted by citations
Items 201–247 of 247
Zhang, S., Ma, H., Wang, L., Wang, M., Li, B., & Liu, J. L. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073058 · Cited by 2
Leeches (Hirudinea) for osteoarthritis
Croft, A. M., Chandra, S., Pérez-Fernández, G. A., & Michalsen, A. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010962 · Cited by 2
Das ABC der medizinischen Statistik
Labenz, J., & Kunz, C. U. (2010) · Der Internist · doi:10.1007/s00108-010-2581-x · Cited by 2
Maduell, F., Moreso, F., Mora-Macià, J., Pons, M., Ramos, R., Carreras, J. S., Soler, J., & Torres, F. (2016) · Nefrología (English Edition) · doi:10.1016/j.nefroe.2016.02.008 · Cited by 2
Behavioural Activation for Depression: a systematic review and controlled clinical trial
Ekers, D. (2011) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · Cited by 2
Antimetabolites for preventing trabeculectomy failure in people undergoing cataract surgery
Thomas, R. E., Crichton, A. C. S., & Thomas, B. C. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010627 · Cited by 2
McChrystal, R., Hanlon, P., Lees, J. S., Phillippo, D., Welton, N. J., Gillies, K., & McAllister, D. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111971 · Cited by 1
An Introduction to Common Systematic Errors in Medical Research
Almasi‐Hashiani, A., Hassanzadeh, J., Eshrati, B., & Khedmati, E. (2012) · Zahedan Journal of Research in Medical Sciences · Cited by 1
Blinding in Biomedical Research
Hróbjartsson, A. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-802460-7.00004-8 · Cited by 1
Wang, J., Yin, W., Zhang, H., Lu, M. S., Gao, W., Yin, L., & Yin, Z. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-022142 · Cited by 1
Edvard Munch (1863-1944) The Scream
Pitman, A. (2011) · The British Journal of Psychiatry · doi:10.1192/bjp.198.1.72 · Cited by 1
Li, L., Hu, S., Zhu, X., Zhu, T., & Li, H. (2020) · Medicine · doi:10.1097/md.0000000000018965 · Cited by 1
“If this was impossible, we will analyze data on an intention to treat (ITT) principle basis, as it is more likely to reflect truth if patients do not participate in follow-up reviews.([13]) We can approximate standard deviation using the standard estimation methods from standard errors, exact P values, or 95% confiden…”
MethodNejstgaard, C. H., Lundh, A., Abdi, S., Clayton, G., Gelle, M. H. A., Laursen, D. R. T., Olorisade, B. K., Savović, J., & Hróbjartsson, A. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1527 · Cited by 2
Defining the Intervention Logic of Access-To-Data Measures: A Problem Analysis
Kim, D. (2021) · Munich studies on innovation and competition · doi:10.1007/978-3-030-86778-2_6 · Cited by 1
Hemmingsson, E., Johansson, K., & Neovius, M. (2010) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.110.006130 · Cited by 1
“This can, for example, be clarified by performing a sensitivity analysis in which baseline data are imputed at follow-up for patients who dropped out or by using multiple imputation for missing values (4).”
BackgroundReichenpfader, U., Thaler, K. J., Richter, A., Thieda, P., & Gartlehner, G. (2012) · Wiener Medizinische Wochenschrift · doi:10.1007/s10354-012-0135-7 · Cited by 1
Sandhu, L. (2013) · TSpace (University of Toronto) · Cited by 1
Acupuncture for patients with lower back pain
下市, 善., 春木, 淳., & 育郎, 若. (2014) · Zen Nihon Shinkyu Gakkai zasshi (Journal of the Japan Society of Acupuncture and Moxibustion) · doi:10.3777/jjsam.64.37
On perhexiline and its application to myocardial protection during cardiac surgery
Drury, N. E. (2012) · University of Birmingham Institutional Research Archive (University of Birmingham)
Das ABC der medizinischen Statistik – Klinische Studien lesen und verstehen
Labenz, J., & Kunz, C. U. (2011) · Spektrum der Augenheilkunde · doi:10.1007/s00717-011-0041-1
Wie man Protokollverletzungen in der Komplementärmedizin richtig behandelt
Lüdtke, R. (2009) · Forschende Komplementärmedizin / Research in Complementary Medicine · doi:10.1159/000261095
Bias and variation in meta-analyses of randomised clinical trials
Nüesch, E. (2010) · Bern Open Repository and Information System (University of Bern) · doi:10.7892/boris.73765
Das ABC der medizinischen Statistik
Labenz, J. (2010) · Der MKG-Chirurg · doi:10.1007/s12285-010-0160-8
Neue Webseite: „Mum hat Brustkrebs“
(2011) · best practice onkologie · doi:10.1007/s11654-011-0303-z
Understanding Research Designs
Ehrenstein, V., & Lash, T. L. (2012) · Elsevier eBooks · doi:10.1016/b978-0-12-384978-6.00007-8
Kahan, B. C., Forbes, G., & Cro, S. (2019) · arXiv (Cornell University)
Hu, S., Li, L., Zhu, X., Fu, Y., Li, H., & Zhu, T. (2020) · doi:10.21203/rs.3.rs-28493/v1
(2010) · Current Opinion in Anaesthesiology · doi:10.1097/aco.0b013e32833f3f68
Reichenpfader, U., Thaler, K. J., Richter, A., Thieda, P., & Gartlehner, G. (2012) · Wiener Medizinische Wochenschrift · doi:10.1007/s10354-012-0057-4
Youn, S., Wong, S. A., Chrystoja, C. C., TOMLINSON, G. A., Wijeysundera, H. C., Bell, C. M., Gagliardi, A. R., Baxter, N. N., Sandhu, L., Takata, J. L., et al. (2020) · doi:10.21203/rs.3.rs-71534/v1
Gravel, J., & Gouin, S. (2010) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2009.12.005
Lee, Z. Y. (2023) · Medical Writing · doi:10.56012/ejyj3404
Sources of Bias in Randomised Controlled Trials
Crombie, I. (2021) · Wiley eBooks · doi:10.1002/9781394322183.ch2
Kudu, E., & Batur, A. (2025) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2025.01.025
Wang, Y., Zhang, R., Huang, Q., Yang, X., Kahaer, A., Rexiti, P., & Xia, M. X. (2025) · Neurosurgical Review · doi:10.1007/s10143-025-03452-x
“%) may inflate treatment effects, increase heterogeneity, and introduce bias from financial incentives, errors, or improper matching, limiting confounder adjustment [4, 31, 45, 51].”
Du, P., Qin, M., Liu, Y., Pang, X., Wang, S., Li, Y., Gao, J., Xu, Z., & Zhang, C. (2025) · International Journal of Stroke · doi:10.1177/17474930251394853
“10 This challenge is especially critical in post-stroke trials, where common single-blind or open-label designs allow participants and/or assessors to know treatment assignments, inevitably introducing conscious or uncon-scious bias into outcome measurements.”
Youn, S., Wong, S. A., Chrystoja, C. C., Tomlinson, G., Wijeysundera, H. C., Bell, C. M., Gagliardi, A. R., Baxter, N. N., Takata, J. L., Sandhu, L., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14746350
Kahan, B. C., Forbes, G., & Cro, S. (2020) · Figshare · doi:10.6084/m9.figshare.12924162
Missingness in the National Cancer Database: A Novel Surrogate for Hepatobiliary Cancer Outcomes
Kemprecos, H. J., Tsai, K. C., Cheah, D. S., Kawano, F., Salirrosas, O., Tigranyan, A., Anderson, A., Polites, G., Cohen, M., Kutlu, O. C., et al. (2026) · Annals of Surgical Oncology · doi:10.1245/s10434-025-18278-6
“45,46 Strategies such as imputation, weighting, or sensitivity analyses are essential to mitigate these biases and ensure robust findings.”
Doughty, H. C., Deja, E., Young, B., Welters, I. D., Shepherd, V., Milosevic, S., O’Hara, K., Carman, J., Sankar, V., Thomas-Jones, E., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-106677
“For ME/ MEO interviews, individuals were eligible if they were (1) ME/MEOs working in NHS Trusts or Health Boards and/ or (2) clinical or research staff involved in recruiting to emergency and critical care studies in adults, working in NHS Trusts or Health Boards.”
Visual performance of phase alteration utilizing sub‐elements lenses for potential myopia management
Lahav-Yacouel, K., Tilia, D., Falk, D., Diec, J., Sha, J., Fedtke, C., Ehrmann, K., & Bakaraju, R. C. (2026) · Optometry and Vision Science · doi:10.1002/ovs2.70023
Le gaspillage évitable de la recherche dans les essais cliniques
Yordanov, Y. (2019) · theses.fr (ABES) · doi:10.70675/130e8831z862fz47f2z8000z2f04c8541788
Understanding research designs
Dudukina, E., Ehrenstein, V., & Lash, T. L. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45863-7.00012-3
Schleimer, T., Kaczorowski, S., Teichert, F., Dias, S., Ehrenbrusthoff, K., Holden, M. A., Lunny, C., Saueressig, T., Henriksen, M., & Belavý, D. L. (2026) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2026.09.002
Loddo, A., Michele, S. D., D’Ancona, G., & Angioni, S. L. (2026) · Facts Views and Vision in ObGyn · doi:10.52054/fvvo.2026.578
Murtaza, A. (2026) · Facts Views and Vision in ObGyn · doi:10.52054/fvvo.2026.559
“2 Therefore, it would be valuable to report the number of failed procedures in each randomised group and clarify whether their exclusion had any potential impact on the observed treatment effects.”
Metadata
- Authors
- Nuesch, E., Trelle, S., Reichenbach, S., Rutjes, A. W. S., Burgi, E., Scherer, M., Altman, D. G., & Juni, P.
- Affiliations
- University of Bern (CH)
- University of Göttingen (DE)
- University of Oxford (GB)
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2009-09-07
- Volume
- 339
- Issue
- sep07 1
- Pages
- b3244-b3244
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- PMC
- PMC2739282
- OpenAlex
- W2057810715
- Cited by
- 248
- References
- 52
- Citations by year
- 2026: 9 · 2025: 11 · 2024: 8 · 2023: 9 · 2022: 7 · 2021: 15 · 2020: 20 · 2019: 11 · 2018: 11 · 2017: 15 · 2016: 15 · 2015: 17 · 2014: 17 · 2013: 20 · 2012: 22
- FWCI
- 1.55
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by-nc)
- Bern Open Repository and Information System (University of Bern) (submittedVersion, cc-by-nc)
- GoeScholar The Publication Server of the Georg-August-Universität Göttingen (Georg-August-Universität Göttingen) (submittedVersion, cc-by)
- Open Access CRIS of the University of Bern
- Funders
- National Science Foundation
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (115067, 104762, 66377, 066377, 66378)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- randomized controlled trials, bias in treatment effect estimates, between-trial heterogeneity, intention-to-treat analysis, osteoarthritis, hip osteoarthritis, knee osteoarthritis, patient-reported pain, complementary therapy, meta-epidemiology
- MeSH terms
- Data Collection, Humans, Pain, Meta-Analysis as Topic, Osteoarthritis, Hip, Bias, Randomized Controlled Trials as Topic, Sample Size, Osteoarthritis, Knee
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
