The DOI matches its Crossref record; 35 of the 36 references it cites were found in a registry or an index.
Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study
Open accessAbstract
Objectives: To study the impact of blinding on estimated treatment effects, and their variation between trials; differentiating between blinding of patients, healthcare providers, and observers; detection bias and performance bias; and types of outcome (the MetaBLIND study). Design: Meta-epidemiological study. Data source: Cochrane Database of Systematic Reviews (2013-14). Eligibility criteria for selecting studies: Meta-analyses with both blinded and non-blinded trials on any topic. Review methods: Blinding status was retrieved from trial publications and authors, and results retrieved automatically from the Cochrane Database of Systematic Reviews. Bayesian hierarchical models estimated the average ratio of odds ratios (ROR), and estimated the increases in heterogeneity between trials, for non-blinded trials (or of unclear status) versus blinded trials. Secondary analyses adjusted for adequacy of concealment of allocation, attrition, and trial size, and explored the association between outcome subjectivity (high, moderate, low) and average bias. An ROR lower than 1 indicated exaggerated effect estimates in trials without blinding. Results: The study included 142 meta-analyses (1153 trials). The ROR for lack of blinding of patients was 0.91 (95% credible interval 0.61 to 1.34) in 18 meta-analyses with patient reported outcomes, and 0.98 (0.69 to 1.39) in 14 meta-analyses with outcomes reported by blinded observers. The ROR for lack of blinding of healthcare providers was 1.01 (0.84 to 1.19) in 29 meta-analyses with healthcare provider decision outcomes (eg, readmissions), and 0.97 (0.64 to 1.45) in 13 meta-analyses with outcomes reported by blinded patients or observers. The ROR for lack of blinding of observers was 1.01 (0.86 to 1.18) in 46 meta-analyses with subjective observer reported outcomes, with no clear impact of degree of subjectivity. Information was insufficient to determine whether lack of blinding was associated with increased heterogeneity between trials. The ROR for trials not reported as double blind versus those that were double blind was 1.02 (0.90 to 1.13) in 74 meta-analyses. Conclusion: No evidence was found for an average difference in estimated treatment effect between trials with and without blinded patients, healthcare providers, or outcome assessors. These results could reflect that blinding is less important than often believed or meta-epidemiological study limitations, such as residual confounding or imprecision. At this stage, replication of this study is suggested and blinding should remain a methodological safeguard in trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, this overemphasizes the assumed effect of blinding in trials, which is not reflected in clinical data [59].”
From Efficacy and Acceptance of Cognitive Behavioral Therapy in Adults with Chronic Fatigue Syndrome: A Meta-analysis. (2024) · cites this paper as [59]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“based on meta-epidemiological studies [22][23][24][25] that found little to no effect of blinding on treatment results among randomised trials (including a review of oral health trials 26 )”
From Management of chronic pain secondary to temporomandibular disorders: a systematic review and network meta-analysis of randomised trials (2023) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, a recent meta-epidemiological study found no evidence that lack of patients’ blinding had an impact on estimate effects.39”
From Efficacy and safety of osteopathic manipulative treatment: an overview of systematic reviews (2022) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Self-citationThe Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,586
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267 · cited 1×
“30 31 Comparisons within trials 13 17-19 provide strong evidence that in specific settings lack of blinding in trials causes considerable bias.”
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049 · cited 2×
“12 We further categorised experimental interventions as alternative/complementary or conventional medicine, to facilitate comparison with a systematic review of trials randomising patients to blinded and unblinded substudies.”
Self-citationHróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2013) · Canadian Medical Association Journal · doi:10.1503/cmaj.120744 · Cited by 527
Self-citationHróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2012) · BMJ · doi:10.1136/bmj.e1119 · Cited by 505 · cited 3×
“17 Similarly, a systematic review of 12 trials randomising patients to blinded and non-blinded substudies reported a pronounced bias due to lack of patient blinding in complementary/ alternative medicine trials with patient reported outcomes, exaggerating effect sizes by 0.56 standard deviations.”
InfluentialSelf-citationEpidemiology and reporting of randomised trials published in PubMed journals
Chan, A.-W., & Altman, D. G. (2005) · The Lancet · doi:10.1016/s0140-6736(05)71879-1 · Cited by 474
Hróbjartsson, A., Emanuelsson, F., Skou Thomsen, A. S., Hilden, J., & Brorson, S. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu115 · Cited by 462
Self-citationSterne, J. A. C., Jüni, P., Schulz, K. F., Altman, D. G., Bartlett, C., & Egger, M. (2002) · Statistics in Medicine · doi:10.1002/sim.1184 · Cited by 382
Self-citationSavović, J., Turner, R. M., Mawdsley, D., Jones, H. E., Beynon, R., Higgins, J. P. T., & Sterne, J. A. C. (2018) · American Journal of Epidemiology · doi:10.1093/aje/kwx344 · Cited by 382
Self-citationRosenthal, R. (1963) · PubMed · Cited by 366
Intentional Ignorance: A History of Blind Assessment and Placebo Controls in Medicine
Kaptchuk, T. J. (1998) · Bulletin of the history of medicine · doi:10.1353/bhm.1998.0159 · Cited by 315
Page, M. J., Higgins, J. P. T., Clayton, G., Sterne, J. A. C., Hróbjartsson, A., & Savović, J. (2016) · PLOS ONE · doi:10.1371/journal.pone.0159267 · Cited by 284 · cited 2×
“7 32 We suggest careful consideration of the risk of confounding and of bias, such as bias due to misclassification of methodological characteristics or due to erroneous identification of treatments as experimental and control, in meta-epidemiological studies.”
Self-citationPragmatic Trials — Guides to Better Patient Care?
Ware, J. H., & Hamel, M. B. (2011) · New England Journal of Medicine · doi:10.1056/nejmp1103502 · Cited by 268 · cited 1×
“29 Convincing theoretical reasons lead us to expect both detection and performance bias in non-blinded trials.”
BackgroundDevereaux, P. J. (2001) · JAMA · doi:10.1001/jama.285.15.2000 · Cited by 258
Higgins, J. P., Ramsay, C., Reeves, B. C., Deeks, J. J., Shea, B., Valentine, J. C., Tugwell, P., & Wells, G. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1056 · Cited by 183 · cited 1×
“7 32 We suggest careful consideration of the risk of confounding and of bias, such as bias due to misclassification of methodological characteristics or due to erroneous identification of treatments as experimental and control, in meta-epidemiological studies.”
BackgroundSelf-citationAkl, E. A., Sun, X., Busse, J. W., Johnston, B. C., Briel, M., Mulla, S., You, J. J., Bassler, D., Lamontagne, F., Vera, C., et al. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.015 · Cited by 179 · cited 1×
“We assessed the blinding status of patients, healthcare providers, and outcome assessors using a modified algorithm derived from that of Akl and colleagues 11 (full details are given in the appendix).”
Blinding in Physical Therapy Trials and Its Association with Treatment Effects
Armijo-Olivo, S. L., Fuentes, J., da Costa, B. R., Saltaji, H., Ha, C., & Cummings, G. G. (2016) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0000000000000521 · Cited by 166 · cited 1×
“7 23 Two additional recent studies partly confirmed this pattern: an analysis of physiotherapy trials 24 found little evidence of an impact of blinding of patients or of outcome assessors, and a study of oral health trials 25 found no evidence of an impact of blinding of outcome assessors, though…”
ResultBerlin, J. A., & Golub, R. M. (2014) · JAMA · doi:10.1001/jama.2014.8167 · Cited by 161
Who is blinded in randomized clinical trials? A study of 200 trials and a survey of authors
Haahr, M. T., & Hróbjartsson, A. (2006) · Clinical Trials · doi:10.1177/1740774506069153 · Cited by 152
Self-citationNüesch, E., Reichenbach, S., Trelle, S., Rutjes, A. W., Liewald, K., Sterchi, R., Altman, D. G., & Jüni, P. (2009) · Arthritis Care & Research · doi:10.1002/art.24894 · Cited by 150
Models for Potentially Biased Evidence in Meta-Analysis using Empirically Based Priors
Welton, N. J., Ades, A. E., Carlin, J. B., Altman, D. G., & Sterne, J. A. C. (2008) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2008.00548.x · Cited by 147 · cited 5×
“14 The model thus enabled us to explore the average degree of bias, and also whether the bias differs (eg, in direction) between meta-analyses (that is, the importance of blinding might depend on the clinical scenario) and between trials (that is, the importance of blinding might depend on factors…”
InfluentialSelf-citationHróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Rasmussen, J. V., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyt270 · Cited by 142 · cited 1×
“30 31 Comparisons within trials 13 17-19 provide strong evidence that in specific settings lack of blinding in trials causes considerable bias.”
Self-citationSaltaji, H., Armijo-Olivo, S. L., Cummings, G. G., Amin, M. S., da Costa, B. R., & Flores‐Mir, C. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0491-0 · Cited by 90
Empirical evaluation of which trial characteristics are associated with treatment effect estimates
Dechartres, A., Trinquart, L., Faber, T., & Ravaud, P. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.005 · Cited by 85
Self-citationMoustgaard, H., Bello, S., Miller, F. G., & Hróbjartsson, A. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.06.020 · Cited by 69
Self-citationMoustgaard, H., Jones, H. E., Savović, J., Clayton, G., Sterne, J. A. C., Higgins, J. P. T., & Hróbjartsson, A. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1392 · Cited by 45
Self-citationSample size calculation for meta‐epidemiological studies
Giraudeau, B., Higgins, J. P. T., Tavernier, E., & Trinquart, L. (2015) · Statistics in Medicine · doi:10.1002/sim.6627 · Cited by 40
Self-citationPatient-Reported Outcome Measures in Neurosurgery: A Review of the Current Literature
Ghimire, P., Hasegawa, H., Kalyal, N., Hurwitz, V., & Ashkan, K. (2017) · Neurosurgery · doi:10.1093/neuros/nyx547 · Cited by 34 · cited 1×
“Clarification is also pertinent to the current debate on the balance between reliability and relevance of unblinded patient reported outcome measures (PROMS), 27 28 and the relative importance of blinded explanatory trials versus unblinded pragmatic trials.”
Bello, S., Moustgaard, H., & Hróbjartsson, A. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.08.002 · Cited by 33 · cited 1×
“Blinding could be lost during the course of a trial, 21 which would tend to attenuate the apparent differences between blinded and non-blinded trials.”
Self-citationInfluence of Priming on Patient-Reported Outcome Measures: A Randomized Controlled Trial.
Claessen, F., Mellema, J. J., Stoop, N., Lubberts, B., Ring, D., & Poolman, R. (2016) · Psychosomatics · doi:10.1016/j.psym.2015.09.005 · Cited by 26 · cited 1×
“Clarification is also pertinent to the current debate on the balance between reliability and relevance of unblinded patient reported outcome measures (PROMS), 27 28 and the relative importance of blinded explanatory trials versus unblinded pragmatic trials.”
Label‐invariant models for the analysis of meta‐epidemiological data
Rhodes, K. M., Mawdsley, D., Turner, R., Jones, H. E., Savović, J., & Higgins, J. P. T. (2017) · Statistics in Medicine · doi:10.1002/sim.7491 · Cited by 13 · cited 6×
“We reanalysed our data with an alternative model not restricted by this assumption, 16 which was not available when we planned our study.”
InfluentialSelf-citationA simple method for converting an odds ratio to effect size for use in meta-analysis
Chinn, S. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001130)19:22
Cited by
Sorted by citations
Minozzi, S., Cinquini, M., Gianola, S., Gonzalez-Lorenzo, M., & Banzi, R. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.06.015 · Cited by 471
Coventry, P. A., Meader, N., Melton, H., Temple, M., Dale, H., Wright, K., Cloitre, M., Karatzias, T., Bisson, J., Roberts, N. P., et al. (2020) · PLoS Medicine · doi:10.1371/journal.pmed.1003262 · Cited by 358
“However, there is compelling meta-epidemiological evidence that estimated treatment effects do not differ between trials with and without blinding of patients, healthcare providers, or outcome assessors [170].”
Butler, C., Dorward, J., Yu, L., Gbinigie, O. A., Hayward, G., Saville, B. R., van Hecke, O., Berry, N., Detry, M. A., Saunders, C. T., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(21)00461-x · Cited by 306
“32 In our study, any potential placebo effect will have probably biased results towards a positive effect of azithromycin, meaning our finding of no effect on self-reported time to first recovery is robust to potential type II error.”
CONSORT 2025 explanation and elaboration: updated guideline for reporting randomised trials
Hopewell, S., Chan, A.-W., Collins, G. S., Hróbjartsson, A., Moher, D., Schulz, K. F., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081124 · Cited by 205
Self-citationMcIsaac, D. I., Kidd, G., Gillis, C., Branje, K., Al-Bayati, M., Baxi, A., Grudzinski, A. L., Boland, L., Veroniki, A.-A., Wolfe, D., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081164 · Cited by 168
“58 59 While blinding in non-drug studies, especially for exercise interventions, is challenging, existing reviews synthesise a number of potential approaches.”
Metadata
- Authors
- Moustgaard, H., Clayton, G. L., Jones, H. E., Boutron, I., Jørgensen, L., Laursen, D. R. T., Olsen, M. F., Paludan-Müller, A., Ravaud, P., Savović, J., Sterne, J. A. C., Higgins, J. P. T., & Hróbjartsson, A.
- Affiliations
- Odense University Hospital (DK)
- University of Bristol (GB)
- Hôtel-Dieu de Paris (FR)
- Cochrane (DK)
- University Hospitals Bristol NHS Foundation Trust (GB)
- University of Southern Denmark (DK)
- Corresponding author
- Moustgaard, Helene
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2020-01-21
- Volume
- 368
- Issue
- 2
- Pages
- l6802
- ISSN
- 1756-1833 (ISSN-L)
- PMC
- PMC7190062
- OpenAlex
- W3001993019
- Cited by
- 300
- References
- 36
- Citations by year
- 2026: 21 · 2025: 45 · 2024: 55 · 2023: 50 · 2022: 49 · 2021: 50 · 2020: 27
- FWCI
- 9.08
- Type
- journal article
- Language
- en
- Update notices
- correction (2020-02-05)
- Open access
- hybrid
- doi.org
- Licence
- Locations
- BMJ (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, cc-by)
- University of Southern Denmark Research Portal (University of Southern Denmark) (acceptedVersion, cc-by-nc)
- HAL (submittedVersion)
- HighWire Press Open Archive (submittedVersion)
- Funders
- University Hospitals Bristol NHS Foundation Trust
- NIHR Bristol Biomedical Research Centre (NF-SI-0617-10145)
- National Institute for Health and Care Research (NF-SI-0611-10168, NF-SI-0616-10111, NIHR200268, NF-SI-0617-10145)
- Department of Health and Social Care
- University of Bristol
- Medical Research Council (MR/K025643/1, MR/M014533/1, MR/L004933/1, MR/L004933/2)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- blinding, randomized controlled trials, treatment effect estimation, detection bias, performance bias, trial heterogeneity, meta-analysis bias, double blinding, allocation concealment, attrition, meta-epidemiology, patient-reported outcomes
- MeSH terms
- Clinical Trials as Topic, Humans, Research Design, Epidemiologic Research Design, Observer Variation, Outcome Assessment, Health Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
