The DOI matches its Crossref record; 35 of the 36 references it cites were found in a registry or an index.
Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study
Open accessAbstract
Objectives: To study the impact of blinding on estimated treatment effects, and their variation between trials; differentiating between blinding of patients, healthcare providers, and observers; detection bias and performance bias; and types of outcome (the MetaBLIND study). Design: Meta-epidemiological study. Data source: Cochrane Database of Systematic Reviews (2013-14). Eligibility criteria for selecting studies: Meta-analyses with both blinded and non-blinded trials on any topic. Review methods: Blinding status was retrieved from trial publications and authors, and results retrieved automatically from the Cochrane Database of Systematic Reviews. Bayesian hierarchical models estimated the average ratio of odds ratios (ROR), and estimated the increases in heterogeneity between trials, for non-blinded trials (or of unclear status) versus blinded trials. Secondary analyses adjusted for adequacy of concealment of allocation, attrition, and trial size, and explored the association between outcome subjectivity (high, moderate, low) and average bias. An ROR lower than 1 indicated exaggerated effect estimates in trials without blinding. Results: The study included 142 meta-analyses (1153 trials). The ROR for lack of blinding of patients was 0.91 (95% credible interval 0.61 to 1.34) in 18 meta-analyses with patient reported outcomes, and 0.98 (0.69 to 1.39) in 14 meta-analyses with outcomes reported by blinded observers. The ROR for lack of blinding of healthcare providers was 1.01 (0.84 to 1.19) in 29 meta-analyses with healthcare provider decision outcomes (eg, readmissions), and 0.97 (0.64 to 1.45) in 13 meta-analyses with outcomes reported by blinded patients or observers. The ROR for lack of blinding of observers was 1.01 (0.86 to 1.18) in 46 meta-analyses with subjective observer reported outcomes, with no clear impact of degree of subjectivity. Information was insufficient to determine whether lack of blinding was associated with increased heterogeneity between trials. The ROR for trials not reported as double blind versus those that were double blind was 1.02 (0.90 to 1.13) in 74 meta-analyses. Conclusion: No evidence was found for an average difference in estimated treatment effect between trials with and without blinded patients, healthcare providers, or outcome assessors. These results could reflect that blinding is less important than often believed or meta-epidemiological study limitations, such as residual confounding or imprecision. At this stage, replication of this study is suggested and blinding should remain a methodological safeguard in trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, this overemphasizes the assumed effect of blinding in trials, which is not reflected in clinical data [59].”
From Efficacy and Acceptance of Cognitive Behavioral Therapy in Adults with Chronic Fatigue Syndrome: A Meta-analysis. (2024) · cites this paper as [59]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“based on meta-epidemiological studies [22][23][24][25] that found little to no effect of blinding on treatment results among randomised trials (including a review of oral health trials 26 )”
From Management of chronic pain secondary to temporomandibular disorders: a systematic review and network meta-analysis of randomised trials (2023) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, a recent meta-epidemiological study found no evidence that lack of patients’ blinding had an impact on estimate effects.39”
From Efficacy and safety of osteopathic manipulative treatment: an overview of systematic reviews (2022) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Self-citationThe Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,586
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267 · cited 1×
“30 31 Comparisons within trials 13 17-19 provide strong evidence that in specific settings lack of blinding in trials causes considerable bias.”
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049 · cited 2×
“12 We further categorised experimental interventions as alternative/complementary or conventional medicine, to facilitate comparison with a systematic review of trials randomising patients to blinded and unblinded substudies.”
Self-citationHróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2013) · Canadian Medical Association Journal · doi:10.1503/cmaj.120744 · Cited by 527
Self-citationHróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2012) · BMJ · doi:10.1136/bmj.e1119 · Cited by 505 · cited 3×
“17 Similarly, a systematic review of 12 trials randomising patients to blinded and non-blinded substudies reported a pronounced bias due to lack of patient blinding in complementary/ alternative medicine trials with patient reported outcomes, exaggerating effect sizes by 0.56 standard deviations.”
InfluentialSelf-citationEpidemiology and reporting of randomised trials published in PubMed journals
Chan, A.-W., & Altman, D. G. (2005) · The Lancet · doi:10.1016/s0140-6736(05)71879-1 · Cited by 474
Hróbjartsson, A., Emanuelsson, F., Skou Thomsen, A. S., Hilden, J., & Brorson, S. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu115 · Cited by 462
Self-citation
Cited by
Sorted by citations
Minozzi, S., Cinquini, M., Gianola, S., Gonzalez-Lorenzo, M., & Banzi, R. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.06.015 · Cited by 471
Coventry, P. A., Meader, N., Melton, H., Temple, M., Dale, H., Wright, K., Cloitre, M., Karatzias, T., Bisson, J., Roberts, N. P., et al. (2020) · PLoS Medicine · doi:10.1371/journal.pmed.1003262 · Cited by 358
“However, there is compelling meta-epidemiological evidence that estimated treatment effects do not differ between trials with and without blinding of patients, healthcare providers, or outcome assessors [170].”
Butler, C., Dorward, J., Yu, L., Gbinigie, O. A., Hayward, G., Saville, B. R., van Hecke, O., Berry, N., Detry, M. A., Saunders, C. T., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(21)00461-x · Cited by 306
“32 In our study, any potential placebo effect will have probably biased results towards a positive effect of azithromycin, meaning our finding of no effect on self-reported time to first recovery is robust to potential type II error.”
CONSORT 2025 explanation and elaboration: updated guideline for reporting randomised trials
Hopewell, S., Chan, A.-W., Collins, G. S., Hróbjartsson, A., Moher, D., Schulz, K. F., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081124 · Cited by 205
Self-citationMcIsaac, D. I., Kidd, G., Gillis, C., Branje, K., Al-Bayati, M., Baxi, A., Grudzinski, A. L., Boland, L., Veroniki, A.-A., Wolfe, D., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081164 · Cited by 168
“58 59 While blinding in non-drug studies, especially for exercise interventions, is challenging, existing reviews synthesise a number of potential approaches.”
Metadata
- Authors
- Moustgaard, H., Clayton, G. L., Jones, H. E., Boutron, I., Jørgensen, L., Laursen, D. R. T., Olsen, M. F., Paludan-Müller, A., Ravaud, P., Savović, J., Sterne, J. A. C., Higgins, J. P. T., & Hróbjartsson, A.
- Affiliations
- Odense University Hospital (DK)
- University of Bristol (GB)
- Hôtel-Dieu de Paris (FR)
- Cochrane (DK)
- University Hospitals Bristol NHS Foundation Trust (GB)
- University of Southern Denmark (DK)
- Corresponding author
- Moustgaard, Helene
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2020-01-21
- Volume
- 368
- Issue
- 2
- Pages
- l6802
- ISSN
- 1756-1833 (ISSN-L)
- PMC
- PMC7190062
- OpenAlex
- W3001993019
- Cited by
- 300
- References
- 36
- Citations by year
- 2026: 21 · 2025: 45 · 2024: 55 · 2023: 50 · 2022: 49 · 2021: 50 · 2020: 27
- FWCI
- 9.08
- Type
- journal article
- Language
- en
- Update notices
- correction (2020-02-05)
- Open access
- hybrid
- doi.org
- Licence
- Locations
- BMJ (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, cc-by)
- University of Southern Denmark Research Portal (University of Southern Denmark) (acceptedVersion, cc-by-nc)
- HAL (submittedVersion)
- HighWire Press Open Archive (submittedVersion)
- Funders
- University Hospitals Bristol NHS Foundation Trust
- NIHR Bristol Biomedical Research Centre (NF-SI-0617-10145)
- National Institute for Health and Care Research (NF-SI-0611-10168, NF-SI-0616-10111, NIHR200268, NF-SI-0617-10145)
- Department of Health and Social Care
- University of Bristol
- Medical Research Council (MR/K025643/1, MR/M014533/1, MR/L004933/1, MR/L004933/2)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- blinding, randomized controlled trials, treatment effect estimation, detection bias, performance bias, trial heterogeneity, meta-analysis bias, double blinding, allocation concealment, attrition, meta-epidemiology, patient-reported outcomes
- MeSH terms
- Clinical Trials as Topic, Humans, Research Design, Epidemiologic Research Design, Observer Variation, Outcome Assessment, Health Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
