Bias in clinical intervention research.
Abstract
Research on bias in clinical trials may help identify some of the reasons why investigators sometimes reach the wrong conclusions about intervention effects. Several quality components for the assessment of bias control have been suggested, but although they seem intrinsically valid, empirical evidence is needed to evaluate their effects on the extent and direction of bias. This narrative review summarizes the findings of methodological studies on the influence of bias in clinical trials. A number of methodological studies suggest that lack of adequate randomization in published trial reports may be associated with more positive estimates of intervention effects. The influence of double-blinding and follow-up is less clear. Several studies have found a significant association between funding sources and pro-industry conclusions. However, the methodological studies also show that bias is difficult to detect and appraise. The extent of bias in individual trials is unpredictable. A-priori exclusion of trials with certain characteristics is not recommended. Appraising bias control in individual trials is necessary to avoid making incorrect conclusions about intervention effects.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The likelihood that the treatment effect reported in a systematic review approximates the truth depends on the validity of the included studies, as certain methodological characteristics may be associated with effect sizes.
“The likelihood that the treatment effect reported in a systematic review approximates the truth depends on the validity of the included studies, as certain methodological characteristics may be associated with effect sizes [87],[88].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [87]
Cites
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769 · see citing works
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634 · see citing works
Bombardier, C., Laine, L., Reicin, A. S., Shapiro, D., Burgos‐Vargas, R., Davis, B. R., Day, R. O., Ferraz, M. B., Hawkey, C. J., Hochberg, M. C., et al. (2000) · New England Journal of Medicine · doi:10.1056/nejm200011233432103 · Cited by 4,014 · see citing works
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560 · see citing works
Assessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309 · cited 3× · see citing works
“Therefore, the use of quality scales is not generally recommended for assessment of bias control in individual trials ( 5 ).”
BackgroundInfluentialPublication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 2× · see citing works
“Potential explanations include violation of the uncertainty principle (76), publication bias (55), and biased interpretation of trial results (75).”
BackgroundUsers' guides to the medical literature
Guyatt, G. H. (1993) · JAMA · doi:10.1001/jama.270.17.2096 · Cited by 2,970 · see citing works
Clinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,918 · see citing works
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907 · see citing works
Meta-analysis: Principles and procedures
Egger, M., Smith, G. D., & Phillips, A. N. (1997) · BMJ · doi:10.1136/bmj.315.7121.1533 · Cited by 2,415 · see citing works
Gluud, L. L., Villumsen, J., & Gluud, C. N. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-11-200112040-00010 · Cited by 2,262 · see citing works
Self-citationWhat is meant by intention to treat analysis? Survey of published randomised controlled trials
Hollis, S., & Campbell, F. (1999) · BMJ · doi:10.1136/bmj.319.7211.670 · Cited by 2,160 · see citing works
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147 · cited 1× · see citing works
“Two systematic reviews found similar associations between funding and proindustry conclusions (73, 74).”
BackgroundThe Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981 · see citing works
Statistics notes: Absence of evidence is not evidence of absence
Altman, D. G., & Bland, J. M. (1995) · BMJ · doi:10.1136/bmj.311.7003.485 · Cited by 1,946 · see citing works
Effectiveness and Efficiency: Random Reflections on Health Services
L'Etang, H. (1973) · Public Health · doi:10.1016/s0033-3506(73)80082-4 · Cited by 1,925 · see citing works
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · see citing works
Scope and Impact of Financial Conflicts of Interest in Biomedical Research: A Systematic Review
Bekelman, J. E., Li, Y., & Gross, C. P. (2003) · JAMA · doi:10.1001/jama.289.4.454 · Cited by 1,816 · see citing works
Risk of Cardiovascular Events Associated with Selective COX-2 Inhibitors
Mukherjee, D. (2001) · JAMA · doi:10.1001/jama.286.8.954 · Cited by 1,741 · see citing works
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628 · see citing works
Why we need observational studies to evaluate the effectiveness of health care
Black, N. (1996) · BMJ · doi:10.1136/bmj.312.7040.1215 · Cited by 1,598 · see citing works
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308 · see citing works
Users' guides to the medical literature : a manual for evidence-based clinical practice
Guyatt, G. H., Rennie, D., Meade, M. O., & Cook, D. J. (2002) · Cited by 1,196 · see citing works
Discrepancies between Meta-Analyses and Subsequent Large Randomized, Controlled Trials
LeLorier, J., Grégoire, G., Benhaddad, A. A., Lapierre, J., & Derderian, F. (1997) · New England Journal of Medicine · doi:10.1056/nejm199708213370806 · Cited by 1,171 · see citing works
Blinding in randomised trials: hiding who got what
Schulz, K. F., & Grimes, D. A. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07816-9 · Cited by 952 · see citing works
Allocation concealment in randomised trials: defending against deciphering
Schulz, K. F., & Grimes, D. A. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07750-4 · Cited by 923 · see citing works
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919 · see citing works
A consumer's guide to subgroup analyses
Oxman, A. D., & Guyatt, G. H. (1992) · Annals of Internal Medicine · doi:10.7326/0003-4819-116-1-78 · Cited by 914 · see citing works
Dickersin, K., Yuan, M.-Q., & Meinert, C. (1992) · Journal of the American Medical Association (JAMA) · doi:10.1001/jama.1992.03480030052036 · Cited by 900 · see citing works
Randomized versus historical controls for clinical trials
Sacks, H., Chalmers, T. C., & Smith, H. (1982) · The American Journal of Medicine · doi:10.1016/0002-9343(82)90815-4 · Cited by 834 · see citing works
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818 · see citing works
Als-Nielsen, B., Chen, W., Gluud, C., & Kjaergard, L. L. (2003) · JAMA · doi:10.1001/jama.290.7.921 · Cited by 746 · see citing works
Users' Guides to the Medical Literature
Guyatt, G. H., Haynes, R. B., Jaeschke, R. Z., Cook, D. J., Green, L. A., Naylor, C. D., Wilson, M. C., Richardson, W. S., & for the Evidence-Based Medicine Working Group (2000) · JAMA · doi:10.1001/jama.284.10.1290 · Cited by 741 · see citing works
An overview of clinical research: the lay of the land
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07283-5 · Cited by 734 · see citing works
Publication bias and clinical trials
Dickersin, K., Chan, S., Chalmersx, T. C., Sacks, H. S., & Smith, H. (1987) · Controlled Clinical Trials · doi:10.1016/0197-2456(87)90155-3 · Cited by 715 · see citing works
Why do we need some large, simple randomized trials?
Yusuf, S., Collins, R., & Peto, R. (1984) · Statistics in Medicine · doi:10.1002/sim.4780030421 · Cited by 700 · see citing works
<i>Meta-analysis:</i> Potentials and promise
Egger, M., & Smith, G. D. (1997) · BMJ · doi:10.1136/bmj.315.7119.1371 · Cited by 646 · see citing works
Sample size slippages in randomised trials: exclusions and the lost and wayward.
Schulz, K. F., & Grimes, D. A. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07882-0 · Cited by 642 · see citing works
Incorporating variations in the quality of individual randomized trials into meta-analysis
Detsky, A. S., Naylor, C. D., O'Rourke, K., McGeer, A. J., & L'Abbé, K. A. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90085-2 · Cited by 640 · see citing works
The Intention-to-Treat Principle: How to Assess the True Effect of Choosing a Medical Treatment
Detry, M. A., & Lewis, R. J. (2014) · JAMA · doi:10.1001/jama.2014.7523 · Cited by 634 · see citing works
Ioannidis, J. P. A. (1998) · JAMA · doi:10.1001/jama.279.4.281 · Cited by 632 · see citing works
Chan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613 · cited 1× · see citing works
“In a survey of authors from 519 randomized trials, 32 percent of responders denied the existence of unreported outcomes, despite evidence to the contrary in their publications (105).”
BackgroundStatistical Power, Sample Size, and Their Reporting in Randomized Controlled Trials
Moher, D. (1994) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1994.03520020048013 · Cited by 608 · see citing works
Subverting Randomization in Controlled Trials
Schulz, K. F. (1995) · JAMA · doi:10.1001/jama.1995.03530180050029 · Cited by 518 · see citing works
Randomisation and baseline comparisons in clinical trials
Altman, D. G., & Doré, C. J. (1990) · The Lancet · doi:10.1016/0140-6736(90)90014-v · Cited by 508 · see citing works
Statistics notes: How to randomise
Altman, D. G., & Bland, J. M. (1999) · BMJ · doi:10.1136/bmj.319.7211.703 · Cited by 506 · see citing works
COX-2 selective non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease
Ray, W. A., Stein, C., Daugherty, J. R., Hall, K., Arbogast, P. G., & Griffin, M. R. (2002) · The Lancet · doi:10.1016/s0140-6736(02)11131-7 · Cited by 503 · see citing works
The uncertainty principle and industry-sponsored research
Djulbegović, B., Lacevic, M., Cantor, A. B., Fields, K. K., Bennett, C. L., Adams, J. R., Kuderer, N. M., & Lyman, G. H. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02605-2 · Cited by 484 · see citing works
Balk, E. M. (2002) · JAMA · doi:10.1001/jama.287.22.2973 · Cited by 478 · see citing works
Khan, A., Warner, H. A., & Brown, W. A. (2000) · Archives of General Psychiatry · doi:10.1001/archpsyc.57.4.311 · Cited by 475 · see citing works
Publication and Related Biases
Song, F., Eastwood, A. J., Gilbody, S. M., Duley, L., & Sutton, A. J. (2001) · SAGE Publications Ltd eBooks · doi:10.4135/9781848608344.n21 · Cited by 470 · see citing works
Cochrane, A. L. (1994) · The British Journal of Psychiatry · doi:10.1017/s000712500007344x · Cited by 441 · see citing works
Randomisation to protect against selection bias in healthcare trials
Odgaard-Jensen, J., Vist, G. E., Timmer, A., Kunz, R., Akl, E. A., Schünemann, H., Briel, M., Nordmann, A. J., Pregno, S., & Oxman, A. D. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000012.pub3 · Cited by 439 · see citing works
Publication and related biases.
Song, Eastwood, Gilbody, Duley, & Sutton (2000) · Health Technology Assessment · doi:10.3310/hta4100 · Cited by 427 · see citing works
Kjaergard, L. L., & Als-Nielsen, B. (2002) · BMJ · doi:10.1136/bmj.325.7358.249 · Cited by 422 · see citing works
Kemeny, M. M., Adak, S., Gray, B. N., Macdonald, J. S., Smith, T., Lipsitz, S. R., Sigurdson, E. R., O'Dwyer, P. J., & Benson, A. B. (2002) · Journal of Clinical Oncology · doi:10.1200/jco.2002.20.6.1499 · Cited by 422 · see citing works
Publication Bias in Editorial Decision Making
Olson, C. M. (2002) · JAMA · doi:10.1001/jama.287.21.2825 · Cited by 395 · see citing works
Khan, A., Khan, S., Kolts, R. L., & Brown, W. A. (2003) · American Journal of Psychiatry · doi:10.1176/appi.ajp.160.4.790 · Cited by 393 · see citing works
Artificial and Bioartificial Support Systems for Acute and Acute-on-Chronic Liver Failure
Gluud, L. L., Liu, J., Als-Nielsen, B. E. T., & Gluud, C. N. (2003) · JAMA · doi:10.1001/jama.289.2.217 · Cited by 376 · see citing works
Self-citationNon-absorbable disaccharides for hepatic encephalopathy: systematic review of randomised trials
Als-Nielsen, B. E. T., Gluud, L. L., & Gluud, C. N. (2004) · BMJ · doi:10.1136/bmj.38048.506134.ee · Cited by 367 · see citing works
Self-citationStatistics notes. Treatment allocation in controlled trials: why randomise?
Altman, D. G., & Bland, J. M. (1999) · BMJ · doi:10.1136/bmj.318.7192.1209 · Cited by 362 · see citing works
Large Trials vs Meta-analysis of Smaller Trials: How Do Their Results Compare?
Cappelleri, J. C. (1996) · JAMA · doi:10.1001/jama.1996.03540160054033 · Cited by 336 · see citing works
Value of Flow Diagrams in Reports of Randomized Controlled Trials
Egger, M. (2001) · JAMA · doi:10.1001/jama.285.15.1996 · Cited by 317 · see citing works
Cholesterol lowering trials in coronary heart disease: Frequency of citation and outcome.
Ravnskov, U. (1992) · BMJ · doi:10.1136/bmj.305.6844.15 · Cited by 313 · see citing works
Soares, H. P., Daniels, S., Kumar, A., Clarke, M. J., Scott, C., Swann, S., & Djulbegović, B. (2004) · BMJ · doi:10.1136/bmj.328.7430.22 · Cited by 310 · see citing works
NIH clinical trials and publication bias.
Dickersin, K., & Min, Y. (1993) · PubMed · Cited by 293 · see citing works
Agreement among reviewers of review articles
Oxman, A. D., Guyatt, G. H., Singer, J., Goldsmith, C. H., Hutchison, B. G., MILNER, R. A., & Streiner, D. L. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90205-n · Cited by 283 · see citing works
Devereaux, P. J. (2001) · JAMA · doi:10.1001/jama.285.15.2000 · Cited by 258 · see citing works
Issues in Comparisons Between Meta-analyses and Large Trials
Ioannidis, J. P. A. (1998) · JAMA · doi:10.1001/jama.279.14.1089 · Cited by 246 · see citing works
Peto, R., & Baigent, C. (1998) · BMJ · doi:10.1136/bmj.317.7167.1170 · Cited by 245 · see citing works
Ascorbic acid for the common cold. A prophylactic and therapeutic trial
Karlowski, T. R. (1975) · JAMA · doi:10.1001/jama.231.10.1038 · Cited by 227 · see citing works
Reference bias in reports of drug trials.
Gøtzsche, P. C. (1987) · BMJ · doi:10.1136/bmj.295.6599.654 · Cited by 224 · see citing works
Owen, C. G., Whincup, P. H., Gilg, J. A., & Cook, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7425.1189 · Cited by 224 · see citing works
Predictive ability of meta-analyses of randomised controlled trials
Villar, J., Carroli, G., & Belizán, J. M. (1995) · The Lancet · doi:10.1016/s0140-6736(95)90646-0 · Cited by 222 · see citing works
Risk of cardiovascular events associated with selective COX-2 inhibitors
Mukherjee, D., Nissen, S. E., & Topol, E. J. (2002) · ACC Current Journal Review · doi:10.1016/s1062-1458(01)00545-1 · Cited by 210 · see citing works
Devereaux, P. J., Choi, P. T.-L., El-Dika, S., Bhandari, M., Montori, V. M., Schünemann, H. J., Garg, A. X., Busse, J. W., Heels-Ansdell, D., Ghali, W. A., et al. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.03.017 · Cited by 209 · see citing works
Pildal, J., Chan, A.-W., Hróbjartsson, A., Forfang, E., Altman, D. G., & Gøtzsche, P. C. (2005) · BMJ · doi:10.1136/bmj.38414.422650.8f · Cited by 209 · see citing works
Systematic bias in traumatic brain injury outcome studies because of loss to follow-up
Corrigan, J. D., Harrison-Felix, C. L., Bogner, J. A., Dijkers, M. P. J. M., Sendroy-Terrill, M., & Whiteneck, G. G. (2003) · Archives of Physical Medicine and Rehabilitation · doi:10.1053/apmr.2003.50093 · Cited by 206 · see citing works
Dickersin, K. (1992) · JAMA · doi:10.1001/jama.267.3.374 · Cited by 193 · see citing works
Krimsky, S., & Rothenberg, L. S. (2001) · Science and Engineering Ethics · doi:10.1007/s11948-001-0041-7 · Cited by 191 · see citing works
Users' Guides to the Medical Literature XVI. How to Use a Treatment Recommendation
Guyatt, G. H. (1999) · JAMA · doi:10.1001/jama.281.19.1836 · Cited by 182 · see citing works
Why we need a broad perspective on meta-analysis
Gotzsche, P. C. (2000) · BMJ · doi:10.1136/bmj.321.7261.585 · Cited by 175 · see citing works
In the dark The reporting of blinding status in randomized controlled trials
Montori, V. M., Bhandari, M., Devereaux, P. J., Manns, B., Ghali, W. A., & Guyatt, G. H. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(02)00446-8 · Cited by 175 · see citing works
Meta-analysis: Unresolved issues and future developments
Smith, G. D., & Egger, M. (1998) · BMJ · doi:10.1136/bmj.316.7126.221 · Cited by 175 · see citing works
Schulz, K. F., Grimes, D. A., Altman, D. G., & Hayes, R. J. (1996) · BMJ · doi:10.1136/bmj.312.7033.742 · Cited by 163 · see citing works
Effects of ascorbic acid on the common cold
Chalmers, T. C. (1975) · The American Journal of Medicine · doi:10.1016/0002-9343(75)90127-8 · Cited by 155 · see citing works
Beyond conflict of interestA common problemBuilding a convincing caseWhat should the BMJ be doing?
Smith, R. W. (1998) · BMJ · doi:10.1136/bmj.317.7154.291 · Cited by 147 · see citing works
How Well Are Randomized Controlled Trials Reported in the Dermatology Literature?
Adetugbo, K., & Williams, H. C. (2000) · Archives of Dermatology · doi:10.1001/archderm.136.3.381 · Cited by 127 · see citing works
Nüesch, R., Schroeder, K., Dieterle, T., Martina, B., & Battegay, E. J. (2001) · BMJ · doi:10.1136/bmj.323.7305.142 · Cited by 120 · see citing works
Randomized clinical trials in Hepatology: Predictors of quality
Gluud, L. L., Nikolova, D., & Gluud, C. N. (1999) · Hepatology · doi:10.1002/hep.510300510 · Cited by 115 · see citing works
Self-citationExposure Measurement in Cohort Studies: The Challenges of Prospective Data Collection
White, E., Hunt, J. R., & Casso, D. (1998) · Epidemiologic Reviews · doi:10.1093/oxfordjournals.epirev.a017971 · Cited by 106 · see citing works
Gluud, L. L., Krogsgaard, K., & Gluud, C. N. (2001) · BMJ · doi:10.1136/bmj.323.7322.1151 · Cited by 96 · see citing works
Self-citationKlein, M. C., Kaczorowski, J. A., Robbins, J. M., Gauthier, R., Jorgensen, S. H., & Joshi, A. K. (1995) · Europe PMC (PubMed Central) · Cited by 93 · see citing works
Subverting randomization in controlled trials
Schulz, K. F. (1995) · JAMA · doi:10.1001/jama.274.18.1456 · Cited by 89 · see citing works
Combining ecological and individual variables to reduce confounding by indication:
Johnston, S. C. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00251-1 · Cited by 74 · see citing works
Citation bias of hepato-biliary randomized clinical trials
Gluud, L. L., & Gluud, C. N. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00513-3 · Cited by 72 · see citing works
Self-citationRandomisation to protect against selection bias in healthcare trials
Kunz, R., Vist, G. E., & Ad, O. (2002) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000012 · Cited by 65 · see citing works
COX-2 selective non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease
Ray, W. A., Stein, C., Daugherty, J. R., Hall, K., Arbogast, P. G., & Griffin, M. R. (2003) · ACC Current Journal Review · doi:10.1016/s1062-1458(02)00991-1 · Cited by 64 · see citing works
Sample size calculations in surgery: Are they done correctly?
Maggard, M. A., O’Connell, J. B., Liu, J. H., Etzioni, D. A., & Ko, C. Y. (2003) · Surgery · doi:10.1067/msy.2003.235 · Cited by 61 · see citing works
Large trials vs meta-analysis of smaller trials: how do their results compare?
Cappelleri, J. C. (1996) · JAMA · doi:10.1001/jama.276.16.1332 · Cited by 60 · see citing works
Quality assessment of reports on clinical trials in the Journal of Hepatology
Gluud, C. N., & Nikolova, D. (1998) · Journal of Hepatology · doi:10.1016/s0168-8278(98)80021-4 · Cited by 58 · see citing works
Survey of claims of no effect in abstracts of Cochrane reviews
Alderson, P., & Chalmers, I. (2003) · BMJ · doi:10.1136/bmj.326.7387.475 · Cited by 57 · see citing works
Design issues in case-control studies
Wacholder, S. (1995) · Statistical Methods in Medical Research · doi:10.1177/096228029500400403 · Cited by 54 · see citing works
DiFranceisco, W. J., Kelly, J. A., Sikkema, K. J., Somlai, A. M., Murphy, D. A., & Stevenson, L. (1998) · American Journal of Public Health · doi:10.2105/ajph.88.7.1068 · Cited by 53 · see citing works
Evening primrose oil for atopic dermatitis
Williams, H. C. (2003) · BMJ · doi:10.1136/bmj.327.7428.1358 · Cited by 50 · see citing works
Discrepancies among megatrials
Furukawa, T. A., Streiner, D. L., & Hori, S. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00250-x · Cited by 49 · see citing works
Schulz, K. F., Chalmers, I., Altman, D. G., Grimes, D. A., & Doré, C. J. (1995) · PubMed · Cited by 49 · see citing works
Quality of reporting of randomised controlled trials in the intensive care literature
Latronico, N., Botteri, M., Minelli, C., Zanotti, C., Bertolini, G., & Candiani, A. (2002) · Intensive Care Medicine · doi:10.1007/s00134-002-1339-x · Cited by 48 · see citing works
Stopping smoking. Do nicotine chewing-gum and postal encouragement add to doctors' advice.
Ia, C., Lyons, E., & Prescott, R. J. (1987) · PubMed · Cited by 48 · see citing works
Allocation Concealment in Clinical Trials
Schulz, K. F., Altman, D. G., & Moher, D. (2002) · JAMA · doi:10.1001/jama.288.19.2406-jlt1120-4-1 · Cited by 47 · see citing works
Kruse, A., Gluud, L. L., Krogsgaard, K., Gluud, C. N., Mortensen, E. L., Gottschau, A., & Bjerg, A. M. (2000) · Controlled Clinical Trials · doi:10.1016/s0197-2456(00)00048-9 · Cited by 47 · see citing works
Self-citationHarrison, J. E. (2003) · Journal of Orthodontics · doi:10.1093/ortho/30.4.309 · Cited by 46 · see citing works
Cochrane, A. L., Maynard, A., & Chalmers, I. (1997) · Medical Entomology and Zoology · Cited by 39 · see citing works
Gluud, L. L., Sørensen, T. I. A., Gøtzsche, P. C., & Gluud, C. N. (2005) · The American Journal of Gastroenterology · doi:10.1111/j.1572-0241.2005.00327.x · Cited by 38 · see citing works
Self-citationValidity of randomized clinical trials in gastroenterology from 1964–2000
Gluud, L. L., Frederiksen, S. L., & Gluud, C. N. (2002) · Gastroenterology · doi:10.1053/gast.2002.32390 · Cited by 34 · see citing works
Self-citation“Negative trials” are positive!
Gluud, C. N. (1998) · Journal of Hepatology · doi:10.1016/s0168-8278(98)80301-2 · Cited by 33 · see citing works
Allocation Concealment in Clinical Trials
Jüni, P., & Egger, M. (2002) · JAMA · doi:10.1001/jama.288.19.2407-jlt1120-4-2 · Cited by 32 · see citing works
Comparing the Methods Used to Compare Antidepressants
Thase, M. E. (2015) · Psychopharmacology Bulletin · doi:10.64719/pb.6844 · Cited by 32 · see citing works
Funding, disease area, and internal validity of hepatobiliary randomized clinical trials
Gluud, L. L., & Gluud, C. N. (2002) · The American Journal of Gastroenterology · doi:10.1111/j.1572-0241.2002.07067.x · Cited by 30 · see citing works
Self-citationEmerging Standards in Statistical Practice
Millis, S. R. (2003) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/01.phm.0000087007.19214.32 · Cited by 29 · see citing works
Huang, W., LaBerge, J. M., Lü, Y., et al. (2002) · Journal of Vascular and Interventional Radiology · doi:10.1016/s1051-0443(07)61717-5 · Cited by 27 · see citing works
Satisfaction of the uncertainty principle in cancer clinical trials: retrospective cohort analysis
Joffe, S., Harrington, D. P., George, S. L., Emanuel, E. J., Budzinski, L., & Weeks, J. C. (2004) · BMJ · doi:10.1136/bmj.38118.685289.55 · Cited by 24 · see citing works
Number and size of randomized trials reported in general health care journals from 1948 to 1997
McDonald, S. J., Westby, M., Clarke, M. J., & Lefebvre, C. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.125 · Cited by 24 · see citing works
Digging for data from the COX-2 trials.
McCormack, J. P., & Rangno, R. (2002) · Europe PMC (PubMed Central) · Cited by 21 · see citing works
Suicide and Antidepressant Treatment
Hirschfeld, R. M. A. (2000) · Archives of General Psychiatry · doi:10.1001/archpsyc.57.4.325 · Cited by 20 · see citing works
British panel bans use of antidepressant to treat children
Abbott, A. (2003) · Nature · doi:10.1038/423792a · Cited by 19 · see citing works
Evidence based medicine in LIVER
Gluud, C. N. (1999) · Liver International · doi:10.1111/j.1478-3231.1999.tb00001.x · Cited by 19 · see citing works
Hutchison, B. G., Oxman, A. D., & Lloyd, S. (1995) · PubMed Central · Cited by 16 · see citing works
Design and quality considerations for randomized controlled trials in systemic sclerosis
Kyriakidi, M., & Ioannidis, J. P. A. (2002) · Arthritis Care & Research · doi:10.1002/art1.10218 · Cited by 16 · see citing works
In vivo assessment of the teratogenic potential of drugs in humans.
STERN, L. O. (1981) · PubMed · Cited by 16 · see citing works
Methods of meta-analysis: an analysis
Koretz, R. L. (2002) · Current Opinion in Clinical Nutrition & Metabolic Care · doi:10.1097/00075197-200209000-00003 · Cited by 16 · see citing works
FDA Alert on Antidepressants for Youth
Mitka, M. (2003) · JAMA · doi:10.1001/jama.290.19.2534 · Cited by 15 · see citing works
FDA questions antidepressant safety for children
Oransky, I. (2003) · The Lancet · doi:10.1016/s0140-6736(03)14787-3 · Cited by 13 · see citing works
Swingler, G. H., & Zwarenstein, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00200-0 · Cited by 13 · see citing works
A Critical Review of Papers from Clinical Cancer Research
Skovlund, E. (1998) · Acta Oncologica · doi:10.1080/028418698430557 · Cited by 10 · see citing works
Dorsch, J. L. (2003) · Europe PMC (PubMed Central) · Cited by 9 · see citing works
University faces up to wartime use of slaves
Abbott, A. (2003) · Nature · doi:10.1038/422792a · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · see citing works
“The likelihood that the treatment effect reported in a systematic review approximates the truth depends on the validity of the included studies, as certain methodological characteristics may be associated with effect sizes [87,88].”
BackgroundLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Italian Journal of Public Health · doi:10.2427/5768 · Cited by 3,374 · see citing works
Genome-wide association studies for complex traits: consensus, uncertainty and challenges
McCarthy, M. I., Abecasis, G. R., Cardon, L. R., Goldstein, D. B., Little, J., Ioannidis, J. P. A., & Hirschhorn, J. N. (2008) · Nature Reviews Genetics · doi:10.1038/nrg2344 · Cited by 2,891 · see citing works
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538 · see citing works
Self-citationBoutron, I., Moher, D., Altman, D. G., Schulz, K. F., Ravaud, P., & for the CONSORT Group* (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-4-200802190-00008 · Cited by 2,292 · see citing works
Bjelakovic, G., Nikolova, D., Gluud, L. L., Simonetti, R. G., & Gluud, C. (2007) · JAMA · doi:10.1001/jama.297.8.842 · Cited by 2,275 · see citing works
Self-citationImproving the reporting of pragmatic trials: An extension of the CONSORT statement
Zwarenstein, M., Treweek, S., Gagnier, J. J., Altman, D. G., Tunis, S., Haynes, B., Oxman, A. D., Moher, D., & groups, F. T. C. A. P. T. I. H. (. (2008) · BMJ · doi:10.1136/bmj.a2390 · Cited by 1,813 · see citing works
Evaluation of networks of randomized trials
Salanti, G., Higgins, J. P., Ades, A., & Ioannidis, J. P. (2008) · Statistical Methods in Medical Research · doi:10.1177/0962280207080643 · Cited by 1,078 · see citing works
Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049 · see citing works
Self-citationKöhler, O., Benros, M. E., Nordentoft, M., Farkouh, M. E., Iyengar, R. L., Mors, O., & Krogh, J. (2014) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1611 · Cited by 941 · see citing works
Bjelakovic, G., Nikolova, D., Gluud, L. L., Simonetti, R. G., & Gluud, C. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007176.pub2 · Cited by 924 · see citing works
Self-citationLittle, J., Higgins, J. P. T., Ioannidis, J. P. A., Moher, D., Gagnon, F., von Elm, E., Khoury, M. J., Cohen, B., Davey-Smith, G., Grimshaw, J., et al. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000022 · Cited by 811 · see citing works
Interpretation of tests of heterogeneity and bias in meta-analysis
Ioannidis, J. P. A. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.00986.x · Cited by 696 · see citing works
Hopewell, S., Clarke, M., Moher, D., Wager, E., Middleton, P., Altman, D. G., Schulz, K. F., & and the CONSORT Group (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050020 · Cited by 674 · see citing works
Savović, J., Turner, R. M., Mawdsley, D., Jones, H. E., Beynon, R., Higgins, J. P. T., & Sterne, J. A. C. (2018) · American Journal of Epidemiology · doi:10.1093/aje/kwx344 · Cited by 382 · see citing works
Reporting methods of blinding in randomized trials assessing nonpharmacological treatments
Boutron, I., Guittet, L., Estellat, C., Moher, D., Hróbjartsson, A., & Ravaud, P. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040061 · Cited by 389 · see citing works
Gourlan, M., Bernard, P., Bortolon, C., Romain, A. J., Lareyre, O., Carayol, M., Ninot, G., & Boiché, J. (2016) · Health Psychology Review · doi:10.1080/17437199.2014.981777 · Cited by 353 · see citing works
Risk of bias versus quality assessment of randomised controlled trials: cross sectional study
Hartling, L., Ospina, M., Liang, Y., Dryden, D. M., Hooton, N., Krebs Seida, J., & Klassen, T. P. (2009) · BMJ · doi:10.1136/bmj.b4012 · Cited by 348 · see citing works
Dechartres, A., Boutron, I., Trinquart, L., Charles, P., & Ravaud, P. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-1-201107050-00006 · Cited by 307 · see citing works
Blood Pressure Response to Transcendental Meditation: A Meta-analysis
Anderson, J. W., Liu, C., & Kryscio, R. J. (2008) · American Journal of Hypertension · doi:10.1038/ajh.2007.65 · Cited by 233 · see citing works
Juul, S., Gluud, C., Simonsen, S., Frandsen, F. W., Kirsch, I., & Jakobsen, J. C. (2021) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111407 · Cited by 135 · see citing works
Prasad, V., Jorgenson, J., Ioannidis, J. P. A., & Cifu, A. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.11.005 · Cited by 90 · see citing works
Empirical evaluation of which trial characteristics are associated with treatment effect estimates
Dechartres, A., Trinquart, L., Faber, T., & Ravaud, P. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.005 · Cited by 85 · see citing works
Hoogeboom, T. J., Kousemaker, M. C., van Meeteren, N. L., Howe, T., Bo, K., Tugwell, P., Ferreira, M., de Bie, R. A., van den Ende, C. H., & Stevens-Lapsley, J. E. (2021) · British Journal of Sports Medicine · doi:10.1136/bjsports-2019-101630 · Cited by 76 · see citing works
Efficacy and safety of acupuncture for idiopathic Parkinson’s disease: a systematic review
Lam, Y. C., Kum, W. F., Durairajan, S. S. K., Lu, J. H., Man, S. C., Xu, M., Zhang, X. F., Huang, X. Z., & Li, M. (2008) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2007.0011 · Cited by 69 · see citing works
Jakobsen, J. C., Hansen, J. L., Simonsen, S., Simonsen, E., & Gluud, C. (2012) · Psychological Medicine · doi:10.1017/s0033291711002236 · Cited by 63 · see citing works
Naci, H., Dias, S., & Ades, A. E. (2014) · BMJ · doi:10.1136/bmj.g5741 · Cited by 61 · see citing works
Who gets recruited in mild traumatic brain injury research?
Luoto, T. M., Tenovuo, O., Kataja, A., Brander, A., Öhman, J., & Iverson, G. L. (2013) · Journal of Neurotrauma · doi:10.1089/neu.2012.2611 · Cited by 62 · see citing works
Lowe, C. M., Wilson, M., Sackley, C., & Barker, K. (2011) · Clinical Rehabilitation · doi:10.1177/0269215510380824 · Cited by 50 · see citing works
Gross, J., Vancampfort, D., Stubbs, B., Gorczynski, P., & Soundy, A. (2016) · Disability and Rehabilitation · doi:10.3109/09638288.2015.1024343 · Cited by 45 · see citing works
A Guide to Planning and Executing a Surgical Randomized Controlled Trial
Chung, K. C., & Burns, P. B. (2008) · The Journal of Hand Surgery · doi:10.1016/j.jhsa.2007.11.027 · Cited by 16 · see citing works
Allocation concealment: a methodological review
Clark, L., Schmidt, U., Tharmanathan, P., Adamson, J., Hewitt, C., & Torgerson, D. (2013) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12032 · Cited by 12 · see citing works
- Authors
- Gluud, L. L.
- Affiliations
- Rigshospitalet (DK)
- Corresponding author
- Gluud, Lise Lotte
- ORCID
- Gluud, Lise Lotte
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2006-01-27
- Volume
- 163
- Issue
- 6
- Pages
- 493-501
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- OpenAlex
- W2127765911
- Cited by
- 636
- References
- 138
- Citations by year
- 2026: 6 · 2025: 32 · 2024: 12 · 2023: 44 · 2022: 15 · 2021: 29 · 2020: 33 · 2019: 40 · 2018: 25 · 2017: 35 · 2016: 24 · 2015: 23 · 2014: 34 · 2013: 40 · 2012: 41
- FWCI
- 12.71
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Pharmacology › Pharmaceutical industry and healthcare
- Keywords
- bias in clinical trials, bias control, double blinding, follow-up, funding sources, industry funding, publication bias, bias assessment, trial quality, intervention effects
- MeSH terms
- Humans, Research Support as Topic, Selection Bias, Odds Ratio, Randomized Controlled Trials as Topic, Publication Bias, Sample Size, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review