Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors

Hróbjartsson, Asbjørn; Thomsen, Ann Sofia Skou; Emanuelsson, Frida; Tendal, Britta; Hilden, Jørgen; Boutron, Isabelle; Ravaud, Philippe; Brorson, Stig

doi:10.1503/cmaj.120744

Cited by 527Meta-analysis and systematic reviewsStatistical Methods in Clinical TrialsReliability and Agreement in Measurement

Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors

Open access

Hróbjartsson, Asbjørn, Thomsen, Ann Sofia Skou, Emanuelsson, Frida, Tendal, Britta, Hilden, Jørgen, Boutron, Isabelle, Ravaud, Philippe, Brorson, Stig·2013·Canadian Medical Association Journal

DOI 10.1503/cmaj.120744PMC PMC3589328

Open this paper in CoChat — free account Open PDF

Abstract

BACKGROUND: Clinical trials are commonly done without blinded outcome assessors despite the risk of bias. We wanted to evaluate the effect of nonblinded outcome assessment on estimated effects in randomized clinical trials with outcomes that involved subjective measurement scales. METHODS: We conducted a systematic review of randomized clinical trials with both blinded and nonblinded assessment of the same measurement scale outcome. We searched PubMed, EMBASE, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, HighWire Press and Google Scholar for relevant studies. Two investigators agreed on the inclusion of trials and the outcome scale. For each trial, we calculated the difference in effect size (i.e., standardized mean difference between nonblinded and blinded assessments). A difference in effect size of less than 0 suggested that nonblinded assessors generated more optimistic estimates of effect. We pooled the differences in effect size using inverse variance random-effects meta-analysis and used metaregression to identify potential reasons for variation. RESULTS: We included 24 trials in our review. The main meta-analysis included 16 trials (involving 2854 patients) with subjective outcomes. The estimated treatment effect was more beneficial when based on nonblinded assessors (pooled difference in effect size -0.23 [95% confidence interval (CI) -0.40 to -0.06]). In relative terms, nonblinded assessors exaggerated the pooled effect size by 68% (95% CI 14% to 230%). Heterogeneity was moderate (I(2) = 46%, p = 0.02) and unexplained by metaregression. INTERPRETATION: We provide empirical evidence for observer bias in randomized clinical trials with subjective measurement scale outcomes. A failure to blind assessors of outcomes in such trials results in a high risk of substantial bias.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1503/cmaj.120744

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Double-blind study design increases methodological rigour by minimising performance and ascertainment biases.
“We considered only double-blind trials because we included placebo in the network meta-analysis, and because this study design increases methodological rigour by minimising performance and ascertainment biases.7”
From Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis (2018) · cites this paper as [7]
The Abstract says blinding outcome assessors helps avoid ascertainment bias, and the Conclusion reports a high risk of bias when assessors are not blinded in trials with subjective measurement-scale outcomes. The paper does not establish that double-blind design minimises performance bias; it also cautions in the Abstract that “double-blind” is an ambiguous term. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Trials at high or unclear risk of bias tend to overestimate benefits and underestimate harms.
“Trials at high or unclear risk of bias tend to overestimate benefits and underestimate harms.”
From Should antidepressants be used for major depressive disorder? (2020) · cites this paper as [38]
Risk of performance bias was evaluated separately for each outcome.
“We evaluated the risk of performance bias separately for each outcome (Hróbjartsson 2013).”
From Diet, physical activity or both for prevention or delay of type 2 diabetes mellitus and its associated complications in people at increased risk of developing type 2 diabetes mellitus (2017) · cites this paper as Hróbjartsson 2013
Risk of detection bias was evaluated separately for each outcome.
“We evaluated the risk of detection bias separately for each outcome (Hróbjartsson 2013).”
From Diet, physical activity or both for prevention or delay of type 2 diabetes mellitus and its associated complications in people at increased risk of developing type 2 diabetes mellitus (2017) · cites this paper as Hróbjartsson 2013

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · see citing works
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373 · see citing works
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
Cochrane Handbook for Systematic Reviews of Interventions
Higgins, J. P. T., Green, S., & Ben Van Den, A. (2020) · International Coaching Psychology Review · doi:10.53841/bpsicpr.2020.15.2.123 · Cited by 5,670 · see citing works
Bilateral Deep Brain Stimulation vs Best Medical Therapy for Patients With Advanced Parkinson Disease A Randomized Controlled Trial
Weaver, F. M. (2009) · JAMA · doi:10.1001/jama.2008.929 · Cited by 1,535 · see citing works
Clozapine Treatment for Suicidality in Schizophrenia<subtitle>International Suicide Prevention Trial (InterSePT)</subtitle>
Meltzer, H. Y. (2003) · Archives of General Psychiatry · doi:10.1001/archpsyc.60.1.82 · Cited by 1,393 · see citing works
Progressive resistance strength training for improving physical function in older adults
Liu, C.-J., & Latham, N. K. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002759.pub2 · Cited by 1,258 · see citing works
Some Problems in Interval Estimation
Fieller, E. C. (1954) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1954.tb00159.x · Cited by 1,159 · see citing works
Influence of Reported Study Design Characteristics on Intervention Effect Estimates From Randomized, Controlled Trials
Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049 · see citing works
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818 · see citing works
Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors
Hróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2012) · BMJ · doi:10.1136/bmj.e1119 · Cited by 487 · see citing works
Self-citation
Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials
Pildal, J., Hrobjartsson, A., Jorgensen, K., Hilden, J., Altman, D., & Gotzsche, P. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym087 · Cited by 472 · see citing works
Self-citation
Reporting methods of blinding in randomized trials assessing nonpharmacological treatments
Boutron, I., Guittet, L., Estellat, C., Moher, D., Hróbjartsson, A., & Ravaud, P. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040061 · Cited by 389 · see citing works
Self-citation
Transmyocardial laser revascularisation compared with continued medical therapy for treatment of refractory angina pectoris: a prospective randomised trial
Burkhoff, D., Schmidt, S. E., Schulman, S. P., MYERS, J. N., Resar, J. R., Becker, L. C., Weiss, J. L., & Jones, J. W. (1999) · The Lancet · doi:10.1016/s0140-6736(99)08113-1 · Cited by 311 · see citing works
Training the Generation X Plastic Surgeon: Dispelling the Myths?
Rohrich, R. J. (2001) · Plastic & Reconstructive Surgery · doi:10.1097/00006534-200111000-00048 · Cited by 277 · see citing works
Ribavirin Aerosol Treatment of Bronchiolitis Associated with Respiratory Syncytial Virus Infection in Infants
Taber, L. H., Knight, V., Gilbert, B. E., McClung, H. W., Wilson, S. Z., Norton, H. J., Thurson, J. M., Gordon, W. H., Atmar, R. L., & Schlaudt, W. R. (1983) · PEDIATRICS · doi:10.1542/peds.72.5.613 · Cited by 272 · see citing works
Physician Interpretations and Textbook Definitions of Blinding Terminology in Randomized Controlled Trials
Devereaux, P. J. (2001) · JAMA · doi:10.1001/jama.285.15.2000 · Cited by 258 · see citing works
A randomized controlled trial evaluating the safety and efficacy of cardiac contractility modulation in advanced heart failure
Kadish, A. H., Nademanee, K., VOLOSIN, K. J., Krueger, S., Neelagaru, S. B., RAVAL, N. Y., OBEL, O. A., Weiner, S., Wish, M., Carson, P. E., et al. (2011) · American Heart Journal · doi:10.1016/j.ahj.2010.10.025 · Cited by 231 · see citing works
A Clinical Trial of Propofol vs Midazolam for Procedural Sedation in a Pediatric Emergency Department
Havel, C. J., Strait, R. T., & HENNES, H. M. (1999) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.1999.tb01180.x · Cited by 193 · see citing works
Progressive resistance strength training for improving physical function in older adults (2009)
(2010) · Australian Journal on Ageing · doi:10.1111/j.1741-6612.2010.00435_1.x · Cited by 186 · see citing works
A Classification of Facial Wrinkles
Lemperle, G., Holmes, R. E., Cohen, S. R., & Lemperle, S. M. (2001) · Plastic and Reconstructive Surgery · doi:10.1097/00006534-200111000-00050 · Cited by 184 · see citing works
A Multicenter Clinical Trial of a Biosynthetic Skin Replacement, Dermagraft-TC, Compared with Cryopreserved Human Cadaver Skin for Temporary Coverage of Excised Burn Wounds
Purdue, G. F., Hunt, J. L., Still, J. M., LAW, E. J., Herndon, D. N., Goldfarb, I. W., Schiller, W. R., Hansbrough, J. F., Hickerson, W. L., Himel, H. N., et al. (1997) · Journal of Burn Care & Rehabilitation · doi:10.1097/00004630-199701000-00009 · Cited by 174 · see citing works
Percutaneous transmyocardial laser revascularisation for severe angina: the PACIFIC randomised trial
Oesterle, S. N., Sanborn, T. A., Ali, N. M., Resar, J. R., Ramee, S. R., Heuser, R. R., Dean, L. S., Knopf, W. D., Schofield, P. M., Schaer, G. L., et al. (2000) · The Lancet · doi:10.1016/s0140-6736(00)03203-7 · Cited by 172 · see citing works
Vaginal Mesh for Prolapse: A Randomized Controlled Trial
Iglesia, C. B., Sokol, A. I., Sokol, E. R., Kudish, B. I., Gutman, R. E., Peterson, J. L., & Shott, S. (2010) · Obstetrics & Gynecology · doi:10.1097/aog.0b013e3181e7d7f8 · Cited by 158 · see citing works
Who is blinded in randomized clinical trials? A study of 200 trials and a survey of authors
Haahr, M. T., & Hróbjartsson, A. (2006) · Clinical Trials · doi:10.1177/1740774506069153 · Cited by 152 · see citing works
Self-citation
Comparison of Smooth-Gel Hyaluronic Acid Dermal Fillers with Cross-linked Bovine Collagen: A Multicenter, Double-Masked, Randomized, Within-Subject Study
Baumann, L., Shamban, A. T., Lupo, M. P., Monheit, G. D., Thomas, J. A., Murphy, D. K., Walker, P., & FOR THE JUVÉDERM VS. ZYPLAST NASOLABIAL FOLD STUDY GROUP (2007) · Dermatologic Surgery · doi:10.1111/j.1524-4725.2007.33352.x · Cited by 150 · see citing works
Radiofrequency Treatment of Turbinate Hypertrophy in Subjects Using Continuous Positive Airway Pressure: A Randomized, Double‐Blind, Placebo‐Controlled Clinical Pilot Trial
Powell, N. B., Zonato, A. I., Weaver, E. M., Li, K. K., Troell, R. J., Riley, R. W., & Guilleminault, C. (2001) · The Laryngoscope · doi:10.1097/00005537-200110000-00023 · Cited by 149 · see citing works
The Clinical Effects of Hyaluronic Acid Ester Nasal Dressing (Merogel) on Intranasal Wound Healing after Functional Endoscopic Sinus Surgery
Miller, R. S., Steward, D. L., Tami, T. A., Sillars, M. J., Seiden, A. M., Shete, M. M., Paskowski, C., & Welge, J. A. (2003) · Otolaryngology · doi:10.1016/s0194-5998(03)00460-1 · Cited by 147 · see citing works
Reporting of Outcomes in Orthopaedic Randomized Trials
Poolman, R. W., Struijs, P. A. A., Krips, R., Sierevelt, I. N., Marti, R. K., Farrokhyar, F., & Bhandari, M. (2007) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.f.00683 · Cited by 128 · see citing works
Artecoll: A Long-Lasting Injectable Wrinkle Filler Material: Report of a Controlled, Randomized, Multicenter Clinical Trial of 251 Subjects
Cohen, S. R., & Holmes, R. E. (2004) · Plastic & Reconstructive Surgery · doi:10.1097/01.prs.0000133169.16467.5f · Cited by 120 · see citing works
Clinical comparison of thiothixene and thioridazine in schizophrenic adolescents
Realmuto, G. M., Erickson, W. D., Yellin, A. M., Hopwood, J. H., & Greenberg, L. M. (1984) · American Journal of Psychiatry · doi:10.1176/ajp.141.3.440 · Cited by 106 · see citing works
Comparison of Topical Therapy for Striae Alba (20% Glycolic Acid/0.05% Tretinoin Versus 20% Glycolic Acid/ 10% L-Ascorbic Acid)
Ash, K., Lord, J., ZUKOWSKL, M., & McDaniel, D. H. (1998) · Dermatologic Surgery · doi:10.1111/j.1524-4725.1998.tb04262.x · Cited by 103 · see citing works
Treatment of Mild, Moderate, and Severe Onychomycosis Using 870- and 930-nm Light Exposure
Landsman, A. S., Robbins, A. H., Angelini, P. F., Wu, C. C., Cook, J. J., Oster, M., & Bornstein, E. (2010) · Journal of the American Podiatric Medical Association · doi:10.7547/1000166 · Cited by 103 · see citing works
Functional outcome of botulinum toxin A injections to the lower limbs in cerebral palsy
Reddihough, D. S., King, J. A., Coleman, G. J., Fosang, A., McCoy, A. T., Thomason, P., & Graham, H. K. (2002) · Developmental Medicine & Child Neurology · doi:10.1017/s0012162201002997 · Cited by 100 · see citing works
A randomized, controlled trial of aerobic exercise in combination with paroxetine in the treatment of panic disorder
Wedekind, D., Broocks, A., Weiss, N., Engel, K. R., Neubert, K., & Bandelow, B. (2010) · The World Journal of Biological Psychiatry · doi:10.3109/15622975.2010.489620 · Cited by 84 · see citing works
Functional outcome of botulinum toxin A injections to the lower limbs in cerebral palsy
Reddihough, D. S., King, J. A., Coleman, G. J., et al. (2002) · Developmental Medicine & Child Neurology · doi:10.1111/j.1469-8749.2002.tb00772.x · Cited by 82 · see citing works
Efficacy, Tolerability and Patient Benefit of Ultrasound-Assisted Wound Treatment versus Surgical Debridement: A Randomized Clinical Study
Herberger, K., Franzke, N., Blome, C., Kirsten, N., & Augustin, M. (2011) · Dermatology · doi:10.1159/000326116 · Cited by 57 · see citing works
Improved interobserver variation after training of doctors in the Neer system: A randomised trial
Brorson, S., Bagger, J., Sylvest, A., & Høbjartsson, A. (2002) · Journal of Bone and Joint Surgery - British Volume · doi:10.1302/0301-620x.84b7.13010 · Cited by 57 · see citing works
Self-citation
Treatment of Mild, Moderate, and Severe Onychomycosis Using 870- and 930-nm Light Exposure
Landsman, A. S., & Robbins, A. H. (2012) · Journal of the American Podiatric Medical Association · doi:10.7547/1020169 · Cited by 53 · see citing works
Clinical Effect of a Single Pulsed Dye Laser Treatment of Fresh Surgical Scars: Randomized Controlled Trial
Alam, M., Pon, K., Laborde, S. V., Kaminer, M. S., Arndt, K. A., & Dover, J. S. (2008) · Dermatologic Surgery · doi:10.1111/1524-4725.2006.32029 · Cited by 49 · see citing works
Cyclophosphamide and plasma exchange in multiple sclerosis
Noseworthy, J. H., Vandervoort, M. K., Mf, P., Ebers, G. C., Shumak, K. H., Seland, T. P., Roberts, R. A., Yetisir, E., Gent, M., & Taylor, D. W. (1991) · The Lancet · doi:10.1016/0140-6736(91)93226-y · Cited by 47 · see citing works
Nonanimal Sourced Hyaluronic Acid–Based Dermal Filler Using a Cohesive Polydensified Matrix Technology is Superior to Bovine Collagen in the Correction of Moderate to Severe Nasolabial Folds
Narins, R. S., Coleman, W. P., DONOFRIO, L. M., Jones, D., Maas, C. S., Monheit, G. D., Kaur, M., Grundy, S. L., Pappert, E. J., & Hardas, B. (2010) · Dermatologic Surgery · doi:10.1111/j.1524-4725.2010.01553.x · Cited by 30 · see citing works
Do Unblinded Assessors Bias Muscle Strength Outcomes in Randomized Controlled Trials of Progressive Resistance Strength Training in Older Adults?
Liu, C.-J., LaValley, M. P., & Latham, N. K. (2010) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0b013e31820174b3 · Cited by 26 · see citing works
STATISTICAL REVIEW AND EVALUATION
Memo, D. D. (2011) · Cited by 24 · see citing works
Cyclophosphamide and plasma exchange in multiple sclerosis
Khatri, B. O., McQuillen, M. P., Hoffmann, R. G., Weiner, H., Hauser, S., Dawson, D., Hafler, D. A., Mackin, G. A., Orav, E. J., Currier, R., et al. (1991) · The Lancet · doi:10.1016/0140-6736(91)92688-x · Cited by 24 · see citing works
Radiographs of hip fractures were digitally altered to mask surgeons to the type of implant without compromising the reliability of quality ratings or making the rating process more difficult
Karanicolas, P. J., Bhandari, M., Walter, S. D., Heels‐Ansdell, D., & Guyatt, G. H. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.05.006 · Cited by 22 · see citing works
Letter: The Combined Use of Needle with Hair Transplanter for Hair Recipient Sites
LEE, D.-Y., CHOI, Y.-L., KIM, M.-G., KIM, J.-A., LEE, K.-J., PARK, J.-H., Cho, H.-J., Yang, J., & Sim, W. (2007) · Dermatologic Surgery · doi:10.1111/j.1524-4725.2007.33026.x · Cited by 20 · see citing works
Vaginal Mesh for Prolapse: A Randomized Controlled Trial
Iglesia, C. B., Sokol, A. I., Gutman, R. E., Sokol, E. R., Kudish, B., & Shott, S. (2010) · Obstetrics and Gynecology · doi:10.1097/aog.0b013e3181ff98b1 · Cited by 18 · see citing works
Improved interobserver variation after training of doctors in the Neer system
Brorson, S., Bagger, J., Sylvest, A., & Høbjartsson, A. (2002) · Journal of Bone and Joint Surgery - British Volume · doi:10.1302/0301-620x.84b7.0840950 · Cited by 17 · see citing works
Self-citation
Cabergolin versus Pergolid
Ulm, G., & Schüler, P. (1999) · Aktuelle Neurologie · doi:10.1055/s-2007-1017651 · Cited by 15 · see citing works
Topic of Histopathology Blinding in Nonclinical Safety Biomarker Qualification Studies
Burkhardt, J. E., Ennulat, D., Pandher, K., Solter, P. F., Troth, S. P., Boyce, R. W., & Zabka, T. S. (2010) · Toxicologic Pathology · doi:10.1177/0192623310371221 · Cited by 15 · see citing works
Vaginal Mesh for Prolapse: A Randomized Controlled Trial
Iglesia, C. B., Sokol, A. I., Gutman, R. E., Sokol, E. R., Kudish, B., & Shott, S. (2010) · Obstetrics and Gynecology · doi:10.1097/aog.0b013e3181ff78e1 · Cited by 11 · see citing works

Flagged first, then by citations

Research works printed but not linked: 4

[2]2. Haahr MT, Hróbjartsson A. Who is blind in randomised clinical trials? An analysis of 200 trials and a survey of authors. Clin Tri- als 2006;3:360-5.
[21]21. Meltzer HY, Alphs L, Green AI, et al. Clozapine treatment for suicidality in schizophrenia: International Suicide Prevention Trial (InterSePT). Arch Gen Psychiatry 2003;60:82-91.
[34]34. Realmuto GM, Erickson WD, Yellin AM, et al. Clinical compar- ison of thiothixene and thioridazine in schizophrenic adoles- cents. Am J Psychiatry 1984;141:440-2.
[41]41. Dodd DC. Blind slide reading or the uninformed versus the informed pathologist. Comments Toxicol 1988;2:88-91.

Sorted by citations

Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis
Cipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., Leucht, S., Ruhe, H. G., Turner, E. H., Higgins, J. P. T., et al. (2018) · The Lancet · doi:10.1016/s0140-6736(17)32802-7 · Cited by 3,541 · see citing works
When and how should multiple imputation be used for handling missing data in randomised clinical trials – a practical guide with flowcharts
Jakobsen, J. C., Gluud, C., Wetterslev, J., & Winkel, P. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0442-1 · Cited by 2,747 · see citing works
Efficacy of treatments for anxiety disorders: a meta-analysis
Bandelow, B., Reitt, M., Röver, C., Michaelis, S., Görlich, Y., & Wedekind, D. (2015) · International Clinical Psychopharmacology · doi:10.1097/yic.0000000000000078 · Cited by 555 · see citing works
Diet, physical activity and behavioural interventions for the treatment of overweight or obese children from the age of 6 to 11 years
Mead, E., Brown, T. J., Rees, K., Azevedo, L. B., Whittaker, V., Jones, D., Olajide, J., Mainardi, G. M., Corpeleijn, E., O’Malley, C. L., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012651 · Cited by 524 · see citing works
Compression stockings to prevent post-thrombotic syndrome: a randomised placebo-controlled trial
Kahn, S. R., Shapiro, S., Wells, P. S., Rodger, M. A., Kovacs, M. J., Anderson, D. R., Tagalakis, V., Houweling, A. H., Ducruet, T., Holcroft, C., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(13)61902-9 · Cited by 517 · see citing works
Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies
Hróbjartsson, A., Emanuelsson, F., Skou Thomsen, A. S., Hilden, J., & Brorson, S. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu115 · Cited by 462 · see citing works
Self-citation
Diet, physical activity and behavioural interventions for the treatment of overweight or obese adolescents aged 12 to 17 years
Al-Khudairy, L., Loveman, E., Colquitt, J. L., Mead, E., Johnson, R. E., Fraser, H., Olajide, J., Murphy, M., Velho, R., O’Malley, C. L., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012691 · Cited by 459 · see citing works
Selective serotonin reuptake inhibitors versus placebo in patients with major depressive disorder. A systematic review with meta-analysis and trial sequential analysis
Jakobsen, J. C., Katakam, K. K., Schou, A., Hellmuth, S. G., Stallknecht, S. E., Leth-Møller, K., Iversen, M., Banke, M. B., Petersen, I. J., Klingenberg, S. L., et al. (2017) · BMC Psychiatry · doi:10.1186/s12888-016-1173-2 · Cited by 366 · see citing works
Comparative Efficacy and Acceptability of 21 Antidepressant Drugs for the Acute Treatment of Adults With Major Depressive Disorder: A Systematic Review and Network Meta-Analysis
Cipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., Leucht, S., Ruhe, H. G., Turner, E. H., Higgins, J. P. T., et al. (2018) · Focus · doi:10.1176/appi.focus.16407 · Cited by 328 · see citing works
Diet, physical activity or both for prevention or delay of type 2 diabetes mellitus and its associated complications in people at increased risk of developing type 2 diabetes mellitus
Hemmingsen, B., Gimenez-Perez, G., Mauricio, D., Roqué i Figuls, M., Metzendorf, M.-I., & Richter, B. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003054.pub4 · Cited by 316 · see citing works
Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study
Moustgaard, H., Clayton, G. L., Jones, H. E., Boutron, I., Jørgensen, L., Laursen, D. R. T., Olsen, M. F., Paludan-Müller, A., Ravaud, P., Savović, J., et al. (2020) · BMJ · doi:10.1136/bmj.l6802 · Cited by 300 · see citing works
Self-citation
Intensive glucose control versus conventional glucose control for type 1 diabetes mellitus
Fullerton, B., Jeitler, K., Seitz, M., Horvath, K., Berghold, A., & Siebenhofer, A. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009122.pub2 · Cited by 299 · see citing works
Empirical Evidence of Study Design Biases in Randomized Trials: Systematic Review of Meta-Epidemiological Studies
Page, M. J., Higgins, J. P. T., Clayton, G., Sterne, J. A. C., Hróbjartsson, A., & Savović, J. (2016) · PLOS ONE · doi:10.1371/journal.pone.0159267 · Cited by 284 · see citing works
Self-citation
Observer bias in animal behaviour research: can we believe what we score, if we score what we believe?
Tuyttens, F., de Graaf, S., Heerkens, J., Jacobs, L., Nalon, E., Ott, S., Stadig, L. M., laer, E. V., & Ampe, B. (2014) · Animal Behaviour · doi:10.1016/j.anbehav.2014.02.007 · Cited by 275 · see citing works
Evidence of experimental bias in the life sciences: why we need blind data recording
Holman, L., Head, M. L., Lanfear, R., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002190 · Cited by 263 · see citing works
Interventions to Promote Fundamental Movement Skills in Childcare and Kindergarten: A Systematic Review and Meta-Analysis
Wick, K., Leeger‐Aschmann, C. S., Monn, N. D., Radtke, T., Ott, L. V., Rebholz, C. E., Cruz, S. S., Gerber, N., Schmutz, E. A., Puder, J. J., et al. (2017) · Sports Medicine · doi:10.1007/s40279-017-0723-1 · Cited by 274 · see citing works
The effects of parent-implemented language interventions on child linguistic outcomes: A meta-analysis
Heidlage, J. K., Cunningham, J. E., Kaiser, A. P., Trivette, C. M., Barton, E. E., Frey, J. R., & Roberts, M. Y. (2019) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2018.12.006 · Cited by 258 · see citing works
Diet, physical activity, and behavioural interventions for the treatment of overweight or obesity in preschool children up to the age of 6 years
Colquitt, J. L., Loveman, E., O’Malley, C. L., Azevedo, L. B., Mead, E., Al-Khudairy, L., Ells, L. J., Metzendorf, M., & Rees, K. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012105 · Cited by 218 · see citing works
Anxious brain networks: A coordinate-based activation likelihood estimation meta-analysis of resting-state functional connectivity studies in anxiety
Xu, J., Van Dam, N. T., Feng, C., Luo, Y., Ai, H., Gu, R., & Xu, P. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.11.005 · Cited by 212 · see citing works
CONSORT 2025 explanation and elaboration: updated guideline for reporting randomised trials
Hopewell, S., Chan, A.-W., Collins, G. S., Hróbjartsson, A., Moher, D., Schulz, K. F., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081124 · Cited by 205 · see citing works
Self-citation
A web-based psychoeducational program for informal caregivers of patients with Alzheimer’s disease: a pilot randomized controlled trial
Cristancho-Lacroix, V., Wrobel, J., Cantegreil-Kallen, I., Dub, T., Rouquette, A., & Rigaud, A.-S. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3717 · Cited by 205 · see citing works
Patient-reported outcome measures (PROMs) as proof of treatment efficacy
Kluzek, S., Dean, B. J. F., & Wartolowska, K. A. (2021) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111573 · Cited by 184 · see citing works
Culturally appropriate health education for people in ethnic minority groups with type 2 diabetes mellitus
Attridge, M., Creamer, J., Ramsden, M., Cannings-John, R. L., & Hawthorne, K. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006424.pub3 · Cited by 180 · see citing works
Placebo response and effect in randomized clinical trials: meta-research with focus on contextual effects
Hafliðadóttir, S. H., Juhl, C. B., Nielsen, S. M., Henriksen, M., Harris, I. A., Bliddal, H., & Christensen, R. (2021) · Trials · doi:10.1186/s13063-021-05454-8 · Cited by 181 · see citing works
Algorithmic fairness in computational medicine
Xu, J., Xiao, Y., Wang, W. H., Ning, Y., Shenkman, E. A., Bian, J., & Wang, F. (2022) · eBioMedicine · doi:10.1016/j.ebiom.2022.104250 · Cited by 170 · see citing works
Should antidepressants be used for major depressive disorder?
Jakobsen, J. C., Gluud, C., & Kirsch, I. (2020) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2019-111238 · Cited by 162 · see citing works
Drug interventions for the treatment of obesity in children and adolescents
Axon, E., Atkinson, G., Richter, B., Metzendorf, M., Baur, L. A., Finer, N., Corpeleijn, E., O’Malley, C. L., & Ells, L. J. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012436 · Cited by 159 · see citing works
World Federation of Societies of Biological Psychiatry (WFSBP) guidelines for treatment of anxiety, obsessive-compulsive and posttraumatic stress disorders – Version 3. Part I: Anxiety disorders
Bandelow, B., Allgulander, C., Baldwin, D. S., Costa, D. L. D. C., Denys, D., Dilbaz, N., Domschke, K., Eriksson, E., Fineberg, N. A., Hättenschwiler, J., et al. (2023) · The World Journal of Biological Psychiatry · doi:10.1080/15622975.2022.2086295 · Cited by 158 · see citing works
Intraoperative neuromonitoring versus visual nerve identification for prevention of recurrent laryngeal nerve injury in adults undergoing thyroid surgery
Cirocchi, R., Arezzo, A., D’Andrea, V., Abraha, I., Popivanov, G., Avenia, N., Gerardi, C., Henry, B. M., Randolph, J., & Barczyński, M. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012483.pub2 · Cited by 159 · see citing works
Mesenchymal stem cells improve locomotor recovery in traumatic spinal cord injury: Systematic review with meta-analyses of rat models
Oliveri, R. S., Bello, S., & Biering‐Sørensen, F. (2013) · Neurobiology of Disease · doi:10.1016/j.nbd.2013.10.014 · Cited by 150 · see citing works
Ultrasound-guided arterial cannulation for paediatrics
Aouad-Maroun, M., Raphael, C. K., Sayyid, S. K., Farah, F., & Akl, E. A. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011364.pub2 · Cited by 144 · see citing works
Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors
Hróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Rasmussen, J. V., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyt270 · Cited by 142 · see citing works
Self-citation
Comparative efficacy and tolerability of pharmacological treatments for the treatment of acute bipolar depression: a systematic review and network meta-analysis
Bahji, A., Ermacora, D., Stephenson, C., Hawken, E. R., & Vazquez, G. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.03.030 · Cited by 139 · see citing works
Psychological interventions for diabetes-related distress in adults with type 2 diabetes mellitus
Chew, B. H., Vos, R. C., Metzendorf, M.-I., Scholten, R. J., & Rutten, G. E. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011469.pub2 · Cited by 138 · see citing works
Blinding in Clinical Trials: Seeing the Big Picture
Monaghan, T. F., Agudelo, C. W., Rahman, S. N., Wein, A. J., Lazar, J., Everaert, K., & Dmochowski, R. R. (2021) · Medicina · doi:10.3390/medicina57070647 · Cited by 143 · see citing works
Parent-only interventions for childhood overweight or obesity in children aged 5 to 11 years
Loveman, E., Al-Khudairy, L., Johnson, R. E., Robertson, W., Colquitt, J. L., Mead, E., Ells, L. J., Metzendorf, M., & Rees, K. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012008 · Cited by 137 · see citing works
Blinding in randomised clinical trials of psychological interventions: a retrospective study of published trial reports
Juul, S., Gluud, C., Simonsen, S., Frandsen, F. W., Kirsch, I., & Jakobsen, J. C. (2021) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111407 · Cited by 135 · see citing works
Methods for assessment of keel bone damage in poultry
Casey-Trott, T. M., Heerkens, J., Petrik, M. T., Regmi, P., Schrader, L., Toscano, M. J., & Widowski, T. M. (2015) · Poultry Science · doi:10.3382/ps/pev223 · Cited by 127 · see citing works
Oral vitamin B 12 versus intramuscular vitamin B 12 for vitamin B 12 deficiency
Wang, H., Li, L., Qin, L., Song, Y., Vidal‐Alaball, J., & Liu, T. H. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004655.pub3 · Cited by 126 · see citing works
Reducing bias in open-label trials where blinded outcome assessment is not feasible: strategies from two randomised trials
Kahan, B. C., Cro, S., Doré, C. J., Bratton, D. J., Rehal, S., Maskell, N. A., Rahman, N., & Jairath, V. (2014) · Trials · doi:10.1186/1745-6215-15-456 · Cited by 123 · see citing works
Nutrition support in hospitalised adults at nutritional risk
Feinberg, J. B., Nielsen, E. E., Korang, S. K., Engell, K. H., Nielsen, M. S., Zhang, K., Didriksen, M., Lund, L., Lindahl, N., Hallum, S., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011598.pub2 · Cited by 133 · see citing works
Catalogue of bias: observer bias
Mahtani, K. R., Spencer, E. A., Brassey, J., & Heneghan, C. J. (2018) · BMJ Evidence-Based Medicine · doi:10.1136/ebmed-2017-110884 · Cited by 118 · see citing works
Long-term effects of weight-reducing drugs in people with hypertension
Siebenhofer, A., Jeitler, K., Horvath, K., Berghold, A., Posch, N. A. S., Meschik, J., & Semlitsch, T. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007654.pub4 · Cited by 116 · see citing works
CogSMART Compensatory Cognitive Training for Traumatic Brain Injury
Twamley, E. W., Thomas, K. R., Gregory, A. M., Jak, A. J., Bondi, M. W., Delis, D. C., & Lohr, J. B. (2014) · Journal of Head Trauma Rehabilitation · doi:10.1097/htr.0000000000000076 · Cited by 111 · see citing works
Intraoperative neuromonitoring versus visual nerve identification for prevention of recurrent laryngeal nerve injury in adults undergoing thyroid surgery
Cirocchi, R., D’Andrea, V., Arezzo, A., Abraha, I., Passera, R., Avenia, N., Randolph, J., & Barczyński, M. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012483 · Cited by 109 · see citing works
IMPRoving Outcomes for children exposed to domestic ViolencE (IMPROVE): an evidence synthesis
Howarth, E., Moore, T. H., Welton, N. J., Lewis, N. V., Stanley, N., MacMillan, H. L., Shaw, A. R. G., Hester, M., Bryden, P. A., & Feder, G. S. (2016) · Public Health Research · doi:10.3310/phr04100 · Cited by 104 · see citing works
Predictors and Causes of Long-Term Mortality in Elderly Patients with Acute Venous Thromboembolism: A Prospective Cohort Study
Faller, N., Limacher, A., Méan, M., Righini, M., Aschwanden, M., Beer, J. H., Frauchiger, B., Osterwalder, J., Kucher, N., Lämmle, B., et al. (2016) · The American Journal of Medicine · doi:10.1016/j.amjmed.2016.09.008 · Cited by 104 · see citing works
Bias in Before–After Studies: Narrative Overview for Anesthesiologists
Ho, A. M. ‐., Phelan, R., Mizubuti, G. B., Murdoch, J. A. C., Wickett, S., Ho, A. K., Shyam, V., & Gilron, I. (2017) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000002705 · Cited by 102 · see citing works
A smartphone application supporting patients with psoriasis improves adherence to topical treatment: a randomized controlled trial
Svendsen, M. T., Andersen, F., Andersen, K. H., Pottegård, A., Johannessen, H., Möller, S., August, B., Feldman, S. R., & Andersen, K. E. (2018) · British Journal of Dermatology · doi:10.1111/bjd.16667 · Cited by 99 · see citing works
Transperitoneal versus retroperitoneal laparoscopic adrenalectomy for adrenal tumours in adults
Arezzo, A., Bullano, A., Cochetti, G. G., Cirocchi, R., Randolph, J., Mearini, E. E., Evangelista, A., Ciccone, G., Bonjer, H. J., & Morino, M. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011668.pub2 · Cited by 94 · see citing works
Statistically Significant Antidepressant-Placebo Differences on Subjective Symptom-Rating Scales Do Not Prove That the Drugs Work: Effect Size and Method Bias Matter!
Hengartner, M. P., & Plöderl, M. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00517 · Cited by 94 · see citing works
Short-acting insulin analogues versus regular human insulin for adults with type 1 diabetes mellitus
Fullerton, B., Siebenhofer, A., Jeitler, K., Horvath, K., Semlitsch, T., Berghold, A., Plank, J., Pieber, T. R., & Gerlach, F. M. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012161 · Cited by 93 · see citing works
Screening for type 2 diabetes mellitus
Peer, N., Balakrishna, Y., & Durão, S. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005266.pub2 · Cited by 92 · see citing works
Lack of blinding of outcome assessors in animal model experiments implies risk of observer bias
Bello, S., Krogsbøll, L. T., Gruber, J., Zhao, Z. J., Fischer, D., & Hróbjartsson, A. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.04.008 · Cited by 91 · see citing works
Self-citation
Empirical evaluation of which trial characteristics are associated with treatment effect estimates
Dechartres, A., Trinquart, L., Faber, T., & Ravaud, P. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.005 · Cited by 85 · see citing works
Self-citation
(Ultra-)long-acting insulin analogues versus NPH insulin (human isophane insulin) for adults with type 2 diabetes mellitus
Semlitsch, T., Engler, J., Siebenhofer, A., Jeitler, K., Berghold, A., & Horvath, K. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005613.pub4 · Cited by 80 · see citing works
Cognitive behavioural therapy halves the risk of repeated suicide attempts: systematic review.
Gøtzsche, P. C., & Gøtzsche, P. K. (2017) · Journal of the Royal Society of Medicine · doi:10.1177/0141076817731904 · Cited by 79 · see citing works
Thyroid surgery for Graves’ disease and Graves’ ophthalmopathy
Liu, Z. W., Masterson, L. M., Fish, B. M., Jani, P., & Chatterjee, K. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010576.pub2 · Cited by 79 · see citing works
Concomitant atrial fibrillation surgery for people undergoing cardiac surgery
Huffman, M. D., Karmali, K. N., Berendsen, M. A., Andrei, A., Kruse, J. T., McCarthy, P. M., & Malaisrie, S. C. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011814.pub2 · Cited by 78 · see citing works
Non-nutritive sweeteners for diabetes mellitus
Lohner, S., de Gaudry, D. K., Toews, I., Ferenci, T., & Meerpohl, J. J. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012885.pub2 · Cited by 75 · see citing works
Resveratrol for adults with type 2 diabetes mellitus
Jeyaraman, M. M., Al‐Yousif, N., Mann, A. S., Dolinsky, V. W., Rabbani, R., Zarychanski, R., & Abou-Setta, A. M. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011919.pub2 · Cited by 74 · see citing works
Subcutaneous rapid-acting insulin analogues for diabetic ketoacidosis
Andrade-Castellanos, C. A., Colunga‐Lozano, L. E., Delgado-Figueroa, N., & González‐Padilla, D. A. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011281.pub2 · Cited by 74 · see citing works
Total or near-total thyroidectomy versus subtotal thyroidectomy for multinodular non-toxic goitre in adults
Cirocchi, R., Trastulli, S., Randolph, J., Guarino, S., Rocco, G. D., Arezzo, A., D’Andrea, V., Santoro, A., Barczyński, M., & Avenia, N. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010370.pub2 · Cited by 74 · see citing works
Mapping ongoing nutrition intervention trials in muscle, sarcopenia, and cachexia: a scoping review of future research
Orsso, C. E., Montes‐Ibarra, M., Findlay, M., van der Meij, B. S., de van der Schueren, M. A. E., Landi, F., Laviano, A., & Prado, C. M. (2022) · Journal of Cachexia, Sarcopenia and Muscle · doi:10.1002/jcsm.12954 · Cited by 73 · see citing works
Feasibility of surgical randomised controlled trials with a placebo arm: a systematic review
Wartolowska, K. A., Collins, P. G. S., Hopewell, S., Judge, A., Dean, B. J. F., Rombach, I., Beard, D. J., & Carr, A. J. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010194 · Cited by 72 · see citing works
Paroxetine versus other anti-depressive agents for depression
Purgato, M., Papola, D., Gastaldon, C., Trespidi, C., Magni, L. R., Rizzo, C., Furukawa, T. A., Watanabe, N., Cipriani, A., & Barbui, C. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006531.pub2 · Cited by 72 · see citing works
Metformin for prevention or delay of type 2 diabetes mellitus and its associated complications in persons at increased risk for the development of type 2 diabetes mellitus
Madsen, K. S., Chi, Y., Metzendorf, M.-I., Richter, B., & Hemmingsen, B. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008558.pub2 · Cited by 70 · see citing works
Long-term effects of weight-reducing drugs in people with hypertension
Siebenhofer, A., Winterholer, S., Jeitler, K., Horvath, K., Berghold, A., Krenn, C., & Semlitsch, T. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007654.pub5 · Cited by 70 · see citing works
Surgery for the treatment of obesity in children and adolescents
Ells, L. J., Mead, E., Atkinson, G., Corpeleijn, E., Roberts, K. E., Viner, R., Baur, L. A., Metzendorf, M., & Richter, B. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011740 · Cited by 70 · see citing works
Subjective and objective outcomes in randomized clinical trials: definitions differed in methods publications and were often absent from trial reports
Moustgaard, H., Bello, S., Miller, F. G., & Hróbjartsson, A. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.06.020 · Cited by 69 · see citing works
Self-citation
Self management interventions for type 2 diabetes in adult people with severe mental illness
McBain, H. B., Mulligan, K., Haddad, M., Flood, C., Jones, J., & Simpson, A. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011361.pub2 · Cited by 67 · see citing works
Zinc supplementation for the prevention of type 2 diabetes mellitus in adults with insulin resistance
Dib, R. E., Gameiro, O. L., Ogata, M. S. P., Módolo, N. S. P., Braz, L. G., Jorge, E. C., do Nascimento, P., & Beletate, V. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005525.pub3 · Cited by 67 · see citing works
Cerebral near-infrared spectroscopy monitoring versus treatment as usual for extremely preterm infants: a protocol for the SafeBoosC randomised clinical phase III trial
Hansen, M. L., Pellicer, A., Gluud, C. N., Dempsey, E. M., Mintzer, J. P., Hyttel-Sørensen, S., Heuchan, A. M., Hagmann, C. F., Ergenekon, E., Dimitriou, G., et al. (2019) · Trials · doi:10.1186/s13063-019-3955-6 · Cited by 65 · see citing works
Ultrasound‐Guided Knee Injections Are More Accurate Than Blind Injections: A Systematic Review of Randomized Controlled Trials
FANG, W. H., Chen, X. T., & Vangsness, C. T. (2021) · Arthroscopy Sports Medicine and Rehabilitation · doi:10.1016/j.asmr.2021.01.028 · Cited by 67 · see citing works
Systematic review of interventions for Indigenous adults with mental and substance use disorders in Australia, Canada, New Zealand and the United States
Leske, S., Harris, M. G., Charlson, F. J., Ferrari, A. J., Baxter, A. J., Logan, J. M., Toombs, M., & Whiteford, H. (2016) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867416662150 · Cited by 65 · see citing works
Lee Silverman Voice Treatment for dysarthria in patients with Parkinson’s disease: a systematic review and meta‐analysis
Yuan, F., Guo, X., Wei, X., Xie, F. X., Zheng, J., Huang, Y., Huang, Z.-H., Chang, Z., Li, H.-X., Guo, Y., et al. (2020) · European Journal of Neurology · doi:10.1111/ene.14399 · Cited by 65 · see citing works
Short-acting insulin analogues versus regular human insulin for adult, non-pregnant persons with type 2 diabetes mellitus
Fullerton, B., Siebenhofer, A., Jeitler, K., Horvath, K., Semlitsch, T., Berghold, A., & Gerlach, F. M. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013228 · Cited by 66 · see citing works
Dipeptidyl-peptidase (DPP)-4 inhibitors and glucagon-like peptide (GLP)-1 analogues for prevention or delay of type 2 diabetes mellitus and its associated complications in people at increased risk for the development of type 2 diabetes mellitus
Hemmingsen, B., Sonne, D. P., Metzendorf, M.-I., & Richter, B. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012204.pub2 · Cited by 63 · see citing works
Metformin monotherapy for adults with type 2 diabetes mellitus
Gnesin, F., Thuesen, A. C. B., Kähler, L. K. A., Madsbad, S., & Hemmingsen, B. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012906.pub2 · Cited by 62 · see citing works
The evolution of assessing bias in Cochrane systematic reviews of interventions: celebrating methodological contributions of the Cochrane Collaboration
Turner, L., Boutron, I., Hróbjartsson, A., Altman, D. G., & Moher, D. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-79 · Cited by 62 · see citing works
Self-citation
Optimal dosing of antibiotics in critically ill patients by using continuous/extended infusions: a systematic review and meta-analysis
Chant, C. A., Leung, A., & Friedrich, J. O. (2013) · Critical Care · doi:10.1186/cc13134 · Cited by 61 · see citing works
Expectancy Effects, Failure of Blinding Integrity, and Placebo Response in Trials of Treatments for Psychiatric Disorders
Huneke, N. T. M., Veronesi, G. F., Garner, M. J., Baldwin, D. S., & Cortese, S. (2025) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2025.0085 · Cited by 61 · see citing works
Insulin secretagogues for prevention or delay of type 2 diabetes mellitus and its associated complications in persons at increased risk for the development of type 2 diabetes mellitus
Hemmingsen, B., Sonne, D. P., Metzendorf, M., & Richter, B. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012151.pub2 · Cited by 72 · see citing works
Hypofractionation for clinically localized prostate cancer
Hickey, B. E., James, M. L., Daly, T., Soh, F.-Y., & Jeffery, M. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011462.pub2 · Cited by 60 · see citing works
Interventions for treatment of COVID-19: Second edition of a living systematic review with meta-analyses and trial sequential analyses (The LIVING Project)
Juul, S., Nielsen, E. E., Feinberg, J. B., Siddiqui, F., Kamp, C. B., Barot, E., Holgersson, J., Nielsen, N., Bentzer, P., Veroniki, A. A., et al. (2021) · PLoS ONE · doi:10.1371/journal.pone.0248132 · Cited by 59 · see citing works
Levothyroxine or minimally invasive therapies for benign thyroid nodules
Bandeira-Echtler, E., Bergerhoff, K., & Richter, B. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004098.pub2 · Cited by 59 · see citing works
Minimally invasive parathyroidectomy guided by intraoperative parathyroid hormone monitoring (IOPTH) and preoperative imaging versus bilateral neck exploration for primary hyperparathyroidism in adults
Ahmadieh, H., Kreidieh, O., Akl, E. A., & Fuleihan, G. E. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010787.pub2 · Cited by 60 · see citing works
Fool’s gold? Why blinded trials are not always best
Anand, Norrie, Bradley, McAuley, & Clarke (2020) · BMJ · doi:10.1136/bmj.l6228 · Cited by 58 · see citing works
EULAR recommendations for the reporting of ultrasound studies in rheumatic and musculoskeletal diseases (RMDs)
Costantino, F., Carmona, L., Boers, M., Backhaus, M., Balint, P. V., Bruyn, G. A. W., Christensen, R., Conaghan, P. G., Ferreira, R. J. O., Garrido-Castro, J. L., et al. (2021) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2020-219816 · Cited by 58 · see citing works
Compelling evidence from meta-epidemiological studies demonstrates overestimation of effects in randomized trials that fail to optimize randomization and blind patients and outcome assessors
Wang, Y., Parpia, S., Couban, R., Wang, Q., Armijo-Olivo, S., Bassler, D., Briel, M., Brignardello-Petersen, R., Gluud, L. L., Keitz, S. A., et al. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.11.001 · Cited by 58 · see citing works
Self-citation
Alpha-glucosidase inhibitors for prevention or delay of type 2 diabetes mellitus and its associated complications in people at increased risk of developing type 2 diabetes mellitus
Moelands, S. V., Lucassen, P. L., Akkermans, R. P., De Grauw, W. J., & Van de Laar, F. A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005061.pub3 · Cited by 57 · see citing works
The effect of perioperative anesthetics for prevention of postoperative delirium on general anesthesia: A network meta-analysis
Cui, Y., Li, G., Cao, R., Luan, L., & Kla, K. M. (2019) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2019.06.028 · Cited by 59 · see citing works
A systematic review of the effects of acupuncture on xerostomia and hyposalivation
Assy, Z., & Brand, H. S. (2018) · BMC Complementary and Alternative Medicine · doi:10.1186/s12906-018-2124-x · Cited by 55 · see citing works
Predicting cognitive behavioral therapy outcome in the outpatient sector based on clinical routine data: A machine learning approach
Hilbert, K., Kunas, S. L., Lueken, U., Kathmann, N., Fydrich, T., & Fehm, L. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103530 · Cited by 54 · see citing works
Qualitative evaluation of artificial intelligence-generated weight management diet plans
Kim, D. W., Park, J. S., Sharma, K., Velazquez, A., Li, L., Ostrominski, J. W., Tran, T., Peréz, R. H. S., & Shin, J. (2024) · Frontiers in Nutrition · doi:10.3389/fnut.2024.1374834 · Cited by 53 · see citing works
Raising Awareness for the Replication Crisis in Clinical Psychology by Focusing on Inconsistencies in Psychotherapy Research: How Much Can We Rely on Published Findings from Efficacy Trials?
Hengartner, M. P. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00256 · Cited by 52 · see citing works
The Effect of Mandala Coloring on State Anxiety: A Systematic Review and Meta-Analysis
Støre, S. J., & Jakobsson, N. (2022) · Art Therapy · doi:10.1080/07421656.2021.2003144 · Cited by 54 · see citing works
Methodological Flaws, Conflicts of Interest, and Scientific Fallacies: Implications for the Evaluation of Antidepressants’ Efficacy and Harm
Hengartner, M. P. (2017) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00275 · Cited by 52 · see citing works
Insulin monotherapy compared with the addition of oral glucose-lowering agents to insulin for people with type 2 diabetes already on insulin therapy and inadequate glycaemic control
Vos, R. C., van Avendonk, M. J., Jansen, H., Goudswaard, A. N. N., van den Donk, M., Gorter, K. J., Kerssen, A., & Rutten, G. E. H. M. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006992.pub2 · Cited by 51 · see citing works
Towards evidence-based medicine for paediatricians
Phillips, B. (2018) · Archives of Disease in Childhood · doi:10.1136/archdischild-2018-315014 · Cited by 50 · see citing works

Empirical evidence of observer bias in randomized clinical trials: updated and expanded analysis of trials with both blinded and non-blinded outcome assessors
Salazar, J., Moustgaard, H., Bracchiglione, J., & Hróbjartsson, A. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111787 · Cited by 22 · see citing works
Self-citation
Empirical Evidence of Observer Bias in Randomised Clinical Trials: Updated and Expanded Analysis of Trials with Blinded and Non-Blinded Outcome AssessorsPreprint
Salazar, J., Moustgaard, H., Bracchiglione, J., & Hróbjartsson, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5141572
Self-citation
Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors
Hróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2012) · BMJ · doi:10.1136/bmj.e1119 · Cited by 487 · see citing works
Self-citation
Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors
Hróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Rasmussen, J. V., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyt270 · Cited by 142 · see citing works
Self-citation
Observer bias in randomised clinical trial with measurement scale outcomes. An analysis of trials with both blind and nonblind outcome assessors. Cochrane Database Syst Rev 2011;133(Suppl 3)
Hróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Boutron, I., Ravaud, P., & Brorson, S. (2011) · Research at the University of Copenhagen (University of Copenhagen)
Self-citation

Authors: Hróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S.
Affiliations: University of Copenhagen (DK)
ORCID: Hróbjartsson, Asbjørn
Thomsen, Ann Sofia Skou
Emanuelsson, Frida
Tendal, Britta
Hilden, Jørgen
Boutron, Isabelle
Ravaud, Philippe
Brorson, Stig
Venue: Canadian Medical Association Journal
Publisher: CMA Impact Inc.
Published: 2013-01-28
Volume: 185
Issue: 4
Pages: E201-E211
ISSN: 0820-3946 (ISSN-L)
1488-2329
DOI: 10.1503/cmaj.120744
PMC: PMC3589328
OpenAlex: W2006175454
Cited by: 527
References: 52
Citations by year: 2026: 30 · 2025: 26 · 2024: 23 · 2023: 54 · 2022: 46 · 2021: 43 · 2020: 35 · 2019: 45 · 2018: 51 · 2017: 31 · 2016: 54 · 2015: 37 · 2014: 29 · 2013: 21 · 2012: 1
FWCI: 9.63
Type: journal article
Language: en
Open access: gold
doi.org
Locations: Canadian Medical Association Journal (publishedVersion)
Europe PMC (PubMed Central) (publishedVersion)
PubMed Central (publishedVersion)
PubMed (publishedVersion)
Research at the University of Copenhagen (University of Copenhagen) (submittedVersion)
Funders: U.S. Department of Veterans Affairs
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Keywords: observer bias, subjective outcomes, risk of bias, effect size estimation, bias in clinical trials
MeSH terms: Double-Blind Method, Observer Variation, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks