Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies
Open accessAbstract
BACKGROUND: Blinding patients in clinical trials is a key methodological procedure, but the expected degree of bias due to nonblinded patients on estimated treatment effects is unknown. METHODS: Systematic review of randomized clinical trials with one sub-study (i.e. experimental vs control) involving blinded patients and another, otherwise identical, sub-study involving nonblinded patients. Within each trial, we compared the difference in effect sizes (i.e. standardized mean differences) between the sub-studies. A difference <0 indicates that nonblinded patients generated a more optimistic effect estimate. We pooled the differences with random-effects inverse variance meta-analysis, and explored reasons for heterogeneity. RESULTS: Our main analysis included 12 trials (3869 patients). The average difference in effect size for patient-reported outcomes was -0.56 (95% confidence interval -0.71 to -0.41), (I(2)=60%, P=0.004), i.e. nonblinded patients exaggerated the effect size by an average of 0.56 standard deviation, but with considerable variation. Two of the 12 trials also used observer-reported outcomes, showing no indication of exaggerated effects due lack of patient blinding. There was a larger effect size difference in 10 acupuncture trials [-0.63 (-0.77 to -0.49)], than in the two non-acupuncture trials [-0.17 (-0.41 to 0.07)]. Lack of patient blinding also increased attrition and use of co-interventions: ratio of control group attrition risk 1.79 (1.18 to 2.70), and ratio of control group co-intervention risk 1.55 (0.99 to 2.43). CONCLUSIONS: This study provides empirical evidence of pronounced bias due to lack of patient blinding in complementary/alternative randomized clinical trials with patient-reported outcomes.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A meta-epidemiological study of blinding showed that unblinded RCTs overestimated the outcome effect by 0.56 standard deviations.
“a meta-epidemiological study of blinding showed that unblinded RCTs overestimated the outcome effect by 0.56 standard deviations”
From Evaluation of reporting quality of abstracts of randomized controlled trials regarding patients with COVID-19 using the CONSORT statement for abstracts (2022) · cites this paper as Hróbjartsson et al., 2014
The Abstract and Discussion report an average exaggeration of 0.56 standard deviations, but specifically for patient-reported outcomes in 12 trials comparing blinded and nonblinded patient sub-studies, with considerable variation. The paper describes itself as a systematic review of those trials, not a meta-epidemiological study of unblinded RCTs generally. (LLM, conf=0.88)
Experimental — accuracy not yet measuredBoth patients and clinicians often correctly guess whether they are in the active drug or placebo group, likely enhancing responses in the drug group and reducing responses in the placebo group.
“Both patients and clinicians often correctly guess whether they are in the active drug or placebo group, likely enhancing responses in the drug group and reducing responses in the placebo group (Chen et al., 2011, 2015; Hróbjartsson et al., 2014; Margraf et al., 1991; Baethge et al., 2013).”
From Revisiting the SSRI vs. placebo debate in the treatment of social anxiety disorder: the role of expectancy effects, neural responsivity, and monoamine transporters (2025) · cites this paper as Hróbjartsson et al., 2014
The Discussion and Bias mechanisms sections report that nonblinded patients produced larger patient-reported treatment effects, apparently through response bias and placebo effects. But the review does not establish that patients or clinicians often correctly guess drug-versus-placebo assignments, or that such guesses enhance drug-group responses while reducing placebo-group responses. Its trials predominantly involved acupuncture, not active-drug comparisons. Page numbers were not available in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe effects of the intervention may have been overestimated because participants were not blinded and substance use was self-reported.
“There is also the possibility that the effects of the intervention were overestimated because participants were not blinded (Hrobjartsson 2014), and substance use was self-reported.”
From Motivational interviewing for substance use reduction (2023) · cites this paper as Hrobjartsson 2014
We are not able to blind patients to their condition, which may lead to bias and exaggeration of the effect.
“We are not able to blind patients to their condition, which may lead to bias and exaggeration of the effect [45].”
Side effects associated with SSRIs can inadvertently reveal participants' group assignments, undermining the blind.
“In particular, side effects associated with SSRIs can inadvertently reveal participants' group assignments, undermining the blind.”
From Revisiting the SSRI vs. placebo debate in the treatment of social anxiety disorder: the role of expectancy effects, neural responsivity, and monoamine transporters (2025) · cites this paper as Hróbjartsson et al., 2014
The Introduction and Discussion address the consequences of patients knowing their treatment assignments, chiefly in complementary or alternative therapy trials. Searches for SSRIs, antidepressants, side effects, and adverse events found no passage saying that drug side effects reveal assignments or break the blind. (LLM, conf=0.73)
Experimental — accuracy not yet measured
Trials at high or unclear risk of bias tend to overestimate benefits and underestimate harms.
“Trials at high or unclear risk of bias tend to overestimate benefits and underestimate harms.”
From Should antidepressants be used for major depressive disorder? (2020) · cites this paper as [36]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers
Thorpe, K. E., Zwarenstein, M., Oxman, A. D., Treweek, S., Furberg, C. D., Altman, D. G., Tunis, S., Bergel, E., Harvey, I., Magid, D. J., et al. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.12.011 · Cited by 1,389 · see citing works
Explanatory and Pragmatic Attitudes in Therapeutical Trials
Schwartz, D., & Lellouch, J. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.01.012 · Cited by 1,253 · see citing works
Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049 · see citing works
Vickers, A. J., Cronin, A. M., Maschino, A. C., Lewith, G. T., MacPherson, H., Foster, N. E., Sherman, K. J., Witt, C. M., & Linde, K. (2012) · Archives of Internal Medicine · doi:10.1001/archinternmed.2012.3654 · Cited by 989 · see citing works
Placebo interventions for all clinical conditions
Hróbjartsson, A., & Gøtzsche, P. C. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003974.pub3 · Cited by 777 · see citing works
Self-citationAcupuncture in patients with osteoarthritis of the knee: a randomised trial
Witt, C. M., Brinkhaus, B., Jena, S., Linde, K., Streng, A., Wagenpfeil, S., Hummelsberger, J., Walther, H., Melchart, D., & Willich, S. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66871-7 · Cited by 660 · see citing works
Acupuncture for Patients With Migraine
Linde, K., Streng, A., Jürgens, S., Hoppe, A., Brinkhaus, B., Witt, C. M., Wagenpfeil, S., Pfaffenrath, V., Hammes, M. G., Weidenhammer, W., et al. (2005) · JAMA · doi:10.1001/jama.293.17.2118 · Cited by 636 · see citing works
Hróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2013) · Canadian Medical Association Journal · doi:10.1503/cmaj.120744 · Cited by 527 · see citing works
Self-citationHróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2012) · BMJ · doi:10.1136/bmj.e1119 · Cited by 487 · see citing works
Self-citationCherkin, D. C., Sherman, K. J., Avins, A. L., Erro, J. H., Ichikawa, L., Barlow, W. E., Delaney, K. M., Hawkes, R. J., Hamilton, L., Pressman, A. R., et al. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.65 · Cited by 439 · see citing works
A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers
Thorpe, K. E., Zwarenstein, M., Oxman, A., Treweek, S. P., Furberg, C. D., Altman, D. G., Tunis, S., Bergel, E. F., Harvey, I. M., Magid, D. J., et al. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.090523 · Cited by 422 · see citing works
Acupuncture and Knee Osteoarthritis
Scharf, H., Mansmann, U., Streitberger, K., Witte, S., Krämer, J., Maier, C., Trampisch, H. J., & Victor, N. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-1-200607040-00005 · Cited by 419 · see citing works
Acupuncture in patients with tension-type headache: randomised controlled trial
Melchart, D., Streng, A., Hoppe, A., Brinkhaus, B., Witt, C. M., Wagenpfeil, S., Pfaffenrath, V., Hammes, M. G., Hummelsberger, J., Irnich, D., et al. (2005) · BMJ · doi:10.1136/bmj.38512.405440.8f · Cited by 415 · see citing works
Madsen, M. V., Gotzsche, P. C., & Hrobjartsson, A. (2009) · BMJ · doi:10.1136/bmj.a3115 · Cited by 372 · see citing works
Electroacupuncture for Control of Myeloablative Chemotherapy–Induced Emesis
Shen, J., Wenger, N. S., Glaspy, J. A., Hays, R. D., Albert, P. S., Choi, C., & Shekelle, P. (2000) · JAMA · doi:10.1001/jama.284.21.2755 · Cited by 350 · see citing works
Yan, T., Hui-Chan, C. W. Y., & Li, L. S. W. (2004) · Stroke · doi:10.1161/01.str.0000149623.24906.63 · Cited by 325 · see citing works
Intentional Ignorance: A History of Blind Assessment and Placebo Controls in Medicine
Kaptchuk, T. J. (1998) · Bulletin of the history of medicine · doi:10.1353/bhm.1998.0159 · Cited by 315 · see citing works
Acupuncture in Patients With Chronic Low Back Pain: A Randomized Controlled Trial
Brinkhaus, B. (2006) · Archives of Internal Medicine · doi:10.1001/.450 · Cited by 285 · see citing works
Osteopathic Manipulative Treatment for Chronic Low Back Pain
Licciardone, J. C., Stoll, S. T., Fulda, K. G., Russo, D. P., Siu, J., Winn, W. B., & Swift, J. N. (2003) · Spine · doi:10.1097/01.brs.0000067110.61471.7d · Cited by 281 · see citing works
Devereaux, P. J. (2001) · JAMA · doi:10.1001/jama.285.15.2000 · Cited by 258 · see citing works
Nuesch, E., Trelle, S., Reichenbach, S., Rutjes, A. W. S., Burgi, E., Scherer, M., Altman, D. G., & Juni, P. (2009) · BMJ · doi:10.1136/bmj.b3244 · Cited by 248 · see citing works
Foster, N. E., Thomas, E., Barlas, P., Hill, J. C., Young, J. A., Mason, E. E., & Hay, E. M. (2007) · BMJ · doi:10.1136/bmj.39280.509803.be · Cited by 242 · see citing works
Suarez-Almazor, M. E., Looney, C., Liu, Y., Cox, V., Pietz, K., Marcus, D. M., & Street, R. L. (2010) · Arthritis Care & Research · doi:10.1002/acr.20225 · Cited by 182 · see citing works
Auricular Acupuncture for Dental Anxiety: A Randomized Controlled Trial
Karst, M., Winterhalter, M., nte, S. M., Francki, B., Hondronikos, A., Eckardt, A. M., Hoy, L., Buhck, H., Bernateck, M., & Fink, M. (2007) · Anesthesia & Analgesia · doi:10.1213/01.ane.0000242531.12722.fd · Cited by 172 · see citing works
What are the main methodological problems in the estimation of placebo effects?
Hróbjartsson, A. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00496-6 · Cited by 171 · see citing works
Self-citationRothstein, R. I., Filipi, C. J., Caca, K., Pruitt, R. E., Mergener, K., Torquati, A., Haber, G. B., Chen, Y., Chang, K. J., Wong, D., et al. (2006) · Gastroenterology · doi:10.1053/j.gastro.2006.07.004 · Cited by 159 · see citing works
PETRI, N., SVANHOLT, P., SOLOW, B., WILDSCHIØDTZ, G., & WINKEL, P. (2008) · Journal of Sleep Research · doi:10.1111/j.1365-2869.2008.00645.x · Cited by 152 · see citing works
Nüesch, E., Reichenbach, S., Trelle, S., Rutjes, A. W., Liewald, K., Sterchi, R., Altman, D. G., & Jüni, P. (2009) · Arthritis Care & Research · doi:10.1002/art.24894 · Cited by 150 · see citing works
Placebo effect studies are susceptible to response bias and to other types of biases
Hróbjartsson, A., Kaptchuk, T. J., & Miller, F. G. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.008 · Cited by 150 · see citing works
Self-citationBafeta, A., Dechartres, A., Trinquart, L., Yavchitz, A., Boutron, I., & Ravaud, P. (2012) · BMJ · doi:10.1136/bmj.e813 · Cited by 129 · see citing works
Schwartz, M. P., Wellink, H., Gooszen, H. G., Conchillo, J. M., Samsom, M., & Smout, A. J. P. M. (2006) · Gut · doi:10.1136/gut.2006.096842 · Cited by 124 · see citing works
Efficacy of P6 acupressure in the treatment of nausea and vomiting during pregnancy
O’Brien, B., Relyea, M. J., & Taerum, T. (1996) · American Journal of Obstetrics and Gynecology · doi:10.1016/s0002-9378(96)70454-4 · Cited by 114 · see citing works
Alkaissi, A., Evertsson, K., Johnsson, V.-A., et al. (2002) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/bf03017897 · Cited by 111 · see citing works
Echinacea for Treating the Common Cold
Barrett, B. P., Brown, R., Rakel, D., Mundt, M. P., Bone, K. M., Barlow, S., & Ewers, T. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-153-12-201012210-00003 · Cited by 109 · see citing works
Wise, R. A., Bartlett, S. J., Brown, E. D., Castro, M., Cohen, R. I., Holbrook, J. T., Irvin, C. G., Rand, C. S., Sockrider, M. M., & Sugar, E. A. (2009) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2009.05.041 · Cited by 107 · see citing works
Blinding in Randomized Clinical Trials: Imposed Impartiality
Hróbjartsson, A., & Boutron, I. (2011) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2011.207 · Cited by 96 · see citing works
Self-citationAcupuncture in patients with acute low back pain: A multicentre randomised controlled clinical trial
Vas, J., Aranda, J. M., Modesto, M., Benítez-Parejo, N., Herrera, A. F., Barquín, D. M., Aguilar, I., Sánchez-Araujo, M., & Rivas‐Ruiz, F. (2012) · Pain · doi:10.1016/j.pain.2012.05.033 · Cited by 90 · see citing works
van der Heijden, G. J. M. G., Leffers, P., Wolters, P., et al. (1999) · Annals of the Rheumatic Diseases · doi:10.1136/ard.58.9.530 · Cited by 90 · see citing works
TENS for children's procedural pain
Lander, J., & Fowler‐Kerry, S. (1993) · Pain · doi:10.1016/0304-3959(93)90133-a · Cited by 80 · see citing works
Alford, J. W., & Fadale, P. D. (2003) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/s0749-8063(03)00734-5 · Cited by 74 · see citing works
Werntoft, E., & Dykes, A. (2001) · PubMed · Cited by 73 · see citing works
Effect of acupressure on nausea and vomiting during pregnancy
Gürkan, Ö. C., & Özkan, H. A. (2007) · Complementary Therapies in Clinical Practice · doi:10.1016/j.ctcp.2007.07.002 · Cited by 69 · see citing works
Fransen, G. A. J., van Marrewijk, C. J., Mujakovic, S., Muris, J. W. M., Laheij, R. J. F., Numans, M. E., de Wit, N. J., Samsom, M., Jansen, J. B. �. �., & Knottnerus, J. A. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-16 · Cited by 65 · see citing works
Moffett, J. A. K., Richardson, P. H., Frost, H., & Osborn, A. (1996) · Pain · doi:10.1016/0304-3959(96)03100-4 · Cited by 65 · see citing works
Liu, C., Xie, J.-P., Wang, L., Zheng, Y., Ma, Z.-B., Yang, H., Xu, C., Shi, G., Li, S., Zhao, J., et al. (2010) · Pain Medicine · doi:10.1111/j.1526-4637.2010.01017.x · Cited by 58 · see citing works
Vas, J., Aranda‐Regules, J. M., Modesto, M., Ramos-Monserrat, M., Barón, M., Aguilar, I., Benítez-Parejo, N., Ramírez-Carmona, C., & Rivas‐Ruiz, F. (2012) · Acupuncture in Medicine · doi:10.1136/acupmed-2012-010261 · Cited by 55 · see citing works
Walach, H., Bösch, H., Lewith, G. T., Naumann, J., Schwarzer, B., Falk, S., Kohls, N., Haraldsson, E., Wiesendanger, H., Nordmann, A. J., et al. (2008) · Psychotherapy and Psychosomatics · doi:10.1159/000116609 · Cited by 53 · see citing works
Isolated Pulmonic Valve Endocarditis Caused by Group B Streptococcus ( Streptococcus agalactiae)
Akram, M., & Khan, I. A. (2001) · Angiology · doi:10.1177/000331970105200309 · Cited by 46 · see citing works
Efficacy versus effectiveness trials
Price, D. B., Hillyer, E. V., & van der Molen, T. (2012) · Current Opinion in Allergy and Clinical Immunology · doi:10.1097/aci.0b013e32835ad059 · Cited by 36 · see citing works
Transcutaneous electrical nerve stimulation following appendicectomy: the placebo effect.
Conn, I. G., Marshall, A. H., Yadav, S., Daly, J. C., & Jaffer, M. (1986) · PubMed Central · Cited by 34 · see citing works
[Acupuncture and pain mechanisms (author's transl)].
Melzack, R. (1976) · PubMed · Cited by 26 · see citing works
Pach, D., Brinkhaus, B., Roll, S., Wegscheider, K., Icke, K., Willich, S. N., & Witt, C. M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0026166 · Cited by 24 · see citing works
Two-Week Topical Treatment with Essaven Gel in Patients with Diabetic Microangiopathy
Cesarone, M. R., INCANDELA, L., Belcaro, G. V., et al. (2001) · Angiology · doi:10.1177/0003319701052003s09 · Cited by 20 · see citing works
Topical Treatment of Venous Microangiopathy in Patients with Venous Ulceration with Essaven Gel
SANCTIS, M. T. D., INCANDELA, L., Belcaro, G., et al. (2001) · Angiology · doi:10.1177/0003319701052003s07 · Cited by 15 · see citing works
Acupuncture in Patients With Chronic Low Back Pain
Brinkhaus, B., Witt, C. M., Jena, S., Linde, K., Streng, A., Wagenpfeil, S., Irnich, D., Walther, H.-U., Melchart, D., & Willich, S. N. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.4.450 · Cited by 14 · see citing works
Association Between Mitral Annular Calcification and Carotid Atheroma
Soylu, M., Demir, A. D., Arda, K., et al. (2001) · Angiology · doi:10.1177/000331970105200307 · Cited by 10 · see citing works
Steinsbekk, A., Bentzen, N., Fønnebø, V. M., et al. (2004) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2004.10.1027 · Cited by 8 · see citing works
Pragmatic trials are randomized and may use a placebo
Thorpe, K. E., Oxman, A. D., Treweek, S. P., et al. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.09.014 · Cited by 7 · see citing works
Tugwell, P., Knottnerus, J. A., & Idzerda, L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.08.005 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [6]6. Hro ´bjartsson A, Emanuelsson F, Skou Thomsen AS et al. Bias due to lack of patient blinding in clinical trials: A systematic review of trials randomizing patients to blind and non-blind sub- studies. Int J Epidemiol 2014;43:1272-83.
- [46]46. Price D, Hillyer EV, van der Molen T. Efficacy versus effective- ness trials: informing guidelines for asthma management. Curr Opin Allergy Clin Immunol 2013;13:50-57.
- [54]54. Melzack R. Acupuncture and pain mechanisms. Anaesthetist 1976;25:204-07.
Cited by
Sorted by citations
Jakobsen, J. C., Gluud, C., Wetterslev, J., & Winkel, P. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0442-1 · Cited by 2,747 · see citing works
Guidelines for reporting meta-epidemiological methodology research
Murad, M. H., & Wang, Z. (2017) · Evidence Based Medicine · doi:10.1136/ebmed-2017-110713 · Cited by 515 · see citing works
Jakobsen, J. C., Katakam, K. K., Schou, A., Hellmuth, S. G., Stallknecht, S. E., Leth-Møller, K., Iversen, M., Banke, M. B., Petersen, I. J., Klingenberg, S. L., et al. (2017) · BMC Psychiatry · doi:10.1186/s12888-016-1173-2 · Cited by 366 · see citing works
Kaner, E. F., Beyer, F. R., Garnett, C., Crane, D., Brown, J., Muirhead, C., Redmore, J., O'Donnell, A., Newham, J. J., de Vocht, F., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011479.pub2 · Cited by 308 · see citing works
Blinding and expectancy confounds in psychedelic randomized controlled trials
Muthukumaraswamy, S. D., Forsyth, A., & Lumley, T. (2021) · Expert Review of Clinical Pharmacology · doi:10.1080/17512433.2021.1933434 · Cited by 324 · see citing works
Defining and Measuring Meditation-Related Adverse Effects in Mindfulness-Based Programs
Britton, W. B., Lindahl, J. R., Cooper, D. J., Canby, N. K., & Palitsky, R. (2021) · Clinical Psychological Science · doi:10.1177/2167702621996340 · Cited by 278 · see citing works
Page, M. J., Higgins, J. P. T., Clayton, G., Sterne, J. A. C., Hróbjartsson, A., & Savović, J. (2016) · PLOS ONE · doi:10.1371/journal.pone.0159267 · Cited by 284 · see citing works
Self-citationMoustgaard, H., Clayton, G. L., Jones, H. E., Boutron, I., Jørgensen, L., Laursen, D. R. T., Olsen, M. F., Paludan-Müller, A., Ravaud, P., Savović, J., et al. (2020) · BMJ · doi:10.1136/bmj.l6802 · Cited by 300 · see citing works
Self-citationPlacebo effect in the treatment of depression and anxiety
Kirsch, I. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00407 · Cited by 205 · see citing works
Should antidepressants be used for major depressive disorder?
Jakobsen, J. C., Gluud, C., & Kirsch, I. (2020) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2019-111238 · Cited by 162 · see citing works
SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials
Hróbjartsson, A., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081660 · Cited by 171 · see citing works
Self-citationBoada, M., López, O. L., Olazarán, J., Núñez, L., Pfeffer, M., Paricio, M., Lorites, J., Piñol‐Ripoll, G., Gámez, J. E., Anaya, F., et al. (2020) · Alzheimer's & Dementia · doi:10.1002/alz.12137 · Cited by 143 · see citing works
Juul, S., Gluud, C., Simonsen, S., Frandsen, F. W., Kirsch, I., & Jakobsen, J. C. (2021) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111407 · Cited by 135 · see citing works
Sharma, S., Hucker, A., Matthews, T., Grohmann, D., & Laws, K. R. (2021) · BMC Psychology · doi:10.1186/s40359-021-00658-8 · Cited by 109 · see citing works
Empirical evaluation of which trial characteristics are associated with treatment effect estimates
Dechartres, A., Trinquart, L., Faber, T., & Ravaud, P. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.005 · Cited by 85 · see citing works
Baclofen for alcohol use disorder
Minozzi, S., Saulle, R., & Rösner, S. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012557.pub2 · Cited by 73 · see citing works
Leske, S., Harris, M. G., Charlson, F. J., Ferrari, A. J., Baxter, A. J., Logan, J. M., Toombs, M., & Whiteford, H. (2016) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867416662150 · Cited by 65 · see citing works
Rice, D., Corace, K., Wolfe, D., Esmaeilisaraji, L., Michaud, A., Grima, A., Austin, B., Douma, R., Barbeau, P., Butler, C., et al. (2020) · PLOS ONE · doi:10.1371/journal.pone.0244401 · Cited by 64 · see citing works
Fool’s gold? Why blinded trials are not always best
Anand, Norrie, Bradley, McAuley, & Clarke (2020) · BMJ · doi:10.1136/bmj.l6228 · Cited by 58 · see citing works
Wang, Y., Parpia, S., Couban, R., Wang, Q., Armijo-Olivo, S., Bassler, D., Briel, M., Brignardello-Petersen, R., Gluud, L. L., Keitz, S. A., et al. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.11.001 · Cited by 58 · see citing works
Motivational interviewing for substance use reduction
Schwenker, R., Dietrich, C. E., Hirpa, S., Nothacker, M., Smedslund, G., Frese, T., & Unverzagt, S. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008063.pub3 · Cited by 56 · see citing works
Zahid, J. A., Grummedal, O., Madsen, M. T., & Gögenur, I. (2020) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2019.10.009 · Cited by 48 · see citing works
Vink, M., & Vink-Niese, A. (2022) · Healthcare · doi:10.3390/healthcare10050898 · Cited by 38 · see citing works
Scott, W., Guildford, B. J., Badenoch, J., Driscoll, E., Chilcot, J., Norton, S., Kemp, H. I., Lee, M. J., Lwanga, J., Boffito, M., et al. (2021) · European Journal of Pain · doi:10.1002/ejp.1762 · Cited by 36 · see citing works
Bello, S., Moustgaard, H., & Hróbjartsson, A. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.08.002 · Cited by 33 · see citing works
Self-citationPatient education for preventing recurrence of venous leg ulcers: a systematic review
Shanley, E., Moore, Z., Patton, D., O’Connor, T., Nugent, L., Budri, A. M., & Avsar, P. (2020) · Journal of Wound Care · doi:10.12968/jowc.2020.29.2.79 · Cited by 27 · see citing works
Pedersen, S. S., Andersen, C. M., Ahm, R., Skovbakke, S. J., Kok, R., Helmark, C., Wiil, U. K., Schmidt, T., Olsen, K. R., Hjelmborg, J., et al. (2021) · BMC Cardiovascular Disorders · doi:10.1186/s12872-020-01801-w · Cited by 24 · see citing works
“Limitations We are not able to blind patients to their condition, which may lead to bias and exaggeration of the effect [45].”
BackgroundTurnin, M.-C., Gourdy, P., Martini, J., Buisson, J.-C., Chauchard, M.-C., Delaunay, J., Schirr-Bonnans, S., Taoui, S., Poncet, M.-F., Cosma, V., et al. (2021) · Diabetes Therapy · doi:10.1007/s13300-021-01095-x · Cited by 20 · see citing works
Pedersen, S. S., Skovbakke, S. J., Wiil, U. K., Schmidt, T., dePont Christensen, R., Brandt, C. J., Sørensen, J., Vinther, M., Larroudé, C. E., Melchior, T. M., et al. (2018) · BMC Cardiovascular Disorders · doi:10.1186/s12872-018-0872-7 · Cited by 18 · see citing works
Blinding and expectancy confounds in psychedelic randomised controlled trials
Muthukumaraswamy, S., Forsyth, A., & Lumley, T. (2021) · doi:10.31234/osf.io/q2hzm · Cited by 15 · see citing works
Yin, Y., Gao, J., Zhang, Y., Zhang, X., Ye, J., & Zhang, J. (2022) · International Journal of Infectious Diseases · doi:10.1016/j.ijid.2022.01.002 · Cited by 10 · see citing works
“…that trials with inadequate or unclear allocation conceal- ent overestimate the treatment effect by up to 7% ( Savovi ́c et al., 012 ), and a meta-epidemiological study of blinding showed that nblinded RCTs overestimated the outcome effect by 0.56 standard eviations ( Hróbjartsson et al., 2014 ).”
BackgroundInfluentialFurmark, T., Wahlstedt, K., & Faria, V. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1531725 · Cited by 1 · see citing works
- Authors
- Hróbjartsson, A., Emanuelsson, F., Skou Thomsen, A. S., Hilden, J., & Brorson, S.
- Affiliations
- University of Copenhagen (DK)
- Corresponding author
- Hróbjartsson, Asbjørn
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2014-05-30
- Volume
- 43
- Issue
- 4
- Pages
- 1272-1283
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- PMC
- PMC4258786
- OpenAlex
- W2147802232
- Cited by
- 462
- References
- 61
- Citations by year
- 2026: 43 · 2025: 41 · 2024: 41 · 2023: 69 · 2022: 36 · 2021: 60 · 2020: 46 · 2019: 42 · 2018: 34 · 2017: 17 · 2016: 22 · 2015: 6 · 2014: 2 · 2013: 2
- FWCI
- 1.98
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Journal of Epidemiology (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Research at the University of Copenhagen (University of Copenhagen) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Neuroscience › Cognitive Neuroscience › Pain Management and Placebo Effect
- Keywords
- treatment effect estimation, patient-reported outcomes, observer-reported outcomes, acupuncture trials, complementary and alternative medicine, attrition, co-interventions
- MeSH terms
- Double-Blind Method, Humans, Bias, Randomized Controlled Trials as Topic, Single-Blind Method
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
