Publication bias in the medical literature: A review by a Canadian research ethics board
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
BACKGROUND: We reviewed the publication record of all protocols submitted to the Capital District Health Authority Research Ethics Board (REB) in Halifax, Nova Scotia, for the period 1995-1996. Because of a heightened awareness of the issue, we hypothesized that there would be less publication bias (a failure to report negative results) and a higher publication rate from completed studies, than previously reported. METHODS: Closed studies were identified from the REB database. Publications were identified by the investigators, requests from sponsors, and a literature review. For each publication, we identified authors, title, journal, number of subjects enrolled, and whether or not the publication was a report of a randomized clinical trial. Comparisons were done using a Student's t test, the Chi-square statistic, or Fisher's exact test as appropriate. RESULTS: From the database of closed studies, 106 remained unpublished, while completed investigations resulted in 84 publications (44% publication rate). The median time to publication was 32.5 months. Publication of statistically significant results occurred in 71/84 trials. Publication of protocols submitted by departments ranged from 91% (anesthesia; 10/11) to 25% [nursing; 2/8 (P<0.05)]. Trials investigating new drugs in Phase 3 or 4 studies were more likely to be published than trials investigating agents in Phase 1 or 2 (P<0.05), and were less likely to be published if sponsored by a pharmaceutical company (P<0.05). CONCLUSIONS: Publication bias continues to be a problem, particularly for early phase investigative studies. Our results suggest that a different approach is required to reduce publication bias. The role that REBs and peer-reviewed journals might play requires further exploration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cytokine Storm in a Phase 1 Trial of the Anti-CD28 Monoclonal Antibody TGN1412
Suntharalingam, G., Perry, M. R., Ward, S., Brett, S. J., Castello-Cortes, A., Brunner, M. D., & Panoskaltsis, N. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa063842 · Cited by 2,164
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147
The Existence of Publication Bias and Risk Factors for Its Occurrence
Dickersin, K. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100097014 · Cited by 1,417
Empirical assessment of effect of publication bias on meta-analyses
Sutton, A. J. (2000) · BMJ · doi:10.1136/bmj.320.7249.1574 · Cited by 1,398
Publication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818
Als-Nielsen, B., Chen, W., Gluud, C., & Kjaergard, L. L. (2003) · JAMA · doi:10.1001/jama.290.7.921 · Cited by 746
Chan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613
Egger, M., & Smith, G. D. (1995) · BMJ · doi:10.1136/bmj.310.6982.752 · Cited by 508
Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Chan, A.-W. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041086 · Cited by 444
Research works printed but not linked: 30
- Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991; 337: 867–72. · DOI · Paper page
- Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 1992; 267: 374–8. · DOI · Paper page
- Antes G, Chalmers I. Under-reporting of clinical trials is unethical. Lancet 2003; 361: 978–9. · DOI
- Chalmers I. Underreporting research is scientific misconduct. JAMA 1990; 263: 1405–8. · DOI · Paper page
- Mann H. Research ethics committees and public dissemination of clinical trial results. Lancet 2002; 360: 406–8. · DOI
- Decullier E, Lheritier V, Chapuis F. Fate of biomedical research protocols and publication bias in France: retrospective cohort study. BMJ 2005; 331: 19. · DOI · Paper page
- Sune-Martin P, Montoro-Ronsano JB. Role of a research ethics committee in follow-up and publication of results. Lancet 2003; 361: 2245–6. · DOI
- De Angelis CD, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA 2004; 292: 1363–4. · DOI · Paper page
- De Angelis CD, Drazen JM, Frizelle FA, et al. Is this clinical trial fully registered? A statement from the International Committee of Medical Journal Editors. JAMA 2005; 293: 2927–9. · DOI
- The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). E6(R1): Good Clinical Practice : Consolidated Guideline. The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) 2006. Available from URL; www.ich.org.
- Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996; 17: 1–12. · DOI · Paper page
- Thomson ISI (Institute for Scientific Information). Journal Citation Report. Selected Scientific Journals 2001. Available from URL; www.genebee.msu.su/ journals.
- Herxheimer A. Open access to industry’s clinically relevant data. BMJ 2004; 329: 64–5. · DOI
- Psaty BM, Furberg CD. COX-2 inhibitors--lessons in drug safety. N Engl J Med 2005; 352: 1133–5. · DOI · Paper page
- Chopra SS. MS JAMA: industry funding of clinical trials: benefit or bias? JAMA. 2003; 290: 113–4. · DOI
- Goodyear M. Learning from the TGN1412 trial. BMJ 2006; 332: 677–8. · DOI
- Zarin DA, Tse T, Ide NC. Trial Registration at ClinicalTrials.gov between May and October 2005. N Engl J Med 2005; 353: 2779–87. · DOI · Paper page
- Steinbrook R. Gag clauses in clinical-trial agreements. N Engl J Med 2005; 352: 2160–2. · DOI
- Olivieri NF, Brittenham GM, McLaren CE, et al. Long-term safety and effectiveness of iron-chelation therapy with deferiprone for thalassemia major. N Engl J Med 1998; 339: 417–23. · DOI · Paper page
- Garland EJ. Facing the evidence: antidepressant treatment in children and adolescents. CMAJ 2004; 170: 489–91. · DOI
- Krzyzanowska MK, Pintilie M, Tannock IF. Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA 2003; 290: 495–501. · DOI · Paper page
- Gilbody SM, Song F, Eastwood AJ, Sutton A. The causes, consequences and detection of publication bias in psychiatry. Acta Psychiatr Scand 2000; 102: 241–9. · DOI · Paper page
- Altman DG. Poor-quality medical research: what can journals do? JAMA 2002; 287: 2765–7. · DOI · Paper page
- Massie BM. Negative, neutral, and discordant results: what should an editor do? J Card Fail 2003; 9: 77–9. · doi:10.1054/jcaf.2003.37
- Shields PG. Publication bias is a scientific problem with adverse ethical outcomes: the case for a section for null results. Cancer Epidemiol Biomarkers Prev 2000; 9: 771–2.
- Yaphe J, Edman R, Knishkowy B, Herman J. The association between funding by commercial interests and study outcome in randomized controlled drug trials. Fam Pract 2001; 18: 565–8. · DOI · Paper page
- Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine--selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 2003; 326: 1171–3. · DOI · Paper page
- Hailey D. Scientific harassment by pharmaceutical companies: time to stop. CMAJ 2000; 162: 212–3.
- Rennie D. Thyroid storm. JAMA 1997; 277: 1238–43. · DOI
- Cho MK, Bero LA. The quality of drug studies published in symposium proceedings. Ann Intern Med 1996; 124: 485–9. · DOI · Paper page
Cited by
Industry sponsorship and research outcome
Lundh, A., Lexchin, J., Mintzes, B., Schroll, J. B., & Bero, L. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000033.pub3 · Cited by 1,697
Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group (2013) · PLoS ONE · doi:10.1371/journal.pone.0066844 · Cited by 1,127
“Thus in total, the search strategy identified 20 eligible empirical studies (Figure 2), of which, four were newly included in this update [21,28,31,32].”
BackgroundResultInfluentialDissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039
Industry sponsorship and research outcome
Lundh, A., Sismondo, S., Lexchin, J., Busuioc, O. A., & Bero, L. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000033.pub2 · Cited by 605
Bjordal, J. M., Lopes-Martins, R. A., Joensen, J., Couppe, C., Ljunggren, A. E., Stergioulas, A., & Johnson, M. I. (2008) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-9-75 · Cited by 300
Effect of Caffeine Ingestion on Muscular Strength and Endurance: A Meta-Analysis
WARREN, G. L., PARK, N. D., MARESCA, R. D., MCKIBANS, K. I., & MILLARD-STAFFORD, M. L. (2010) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e3181cabbd8 · Cited by 295
van Uffelen, J. G. Z., Chinapaw, M. J. M., van Mechelen, W., & Hopman-Rock, M. (2008) · British Journal of Sports Medicine · doi:10.1136/bjsm.2007.044735 · Cited by 284
Schmucker, C., Schell, L. K., Portalupi, S., Oeller, P., Cabrera, L., Bassler, D., Schwarzer, G., Scherer, R. W., Antes, G., von Elm, E., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114023 · Cited by 269
Comparison of protocols and registry entries to published reports for randomised controlled trials
Dwan, K., Altman, D. G., Cresswell, L., Blundell, M., Gamble, C. L., & Williamson, P. R. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000031.pub2 · Cited by 243
Reporting research results: a moral obligation for all researchers.
Moher, D. (2007) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/bf03022653 · Cited by 60
A review of the literature on ethical issues related to scientific authorship
Hosseini, M., & Gordijn, B. (2020) · Accountability in Research · doi:10.1080/08989621.2020.1750957 · Cited by 38
A retrospective analysis of dissemination biases in the brief alcohol intervention literature.
Tanner-Smith, E. E., & Polanin, J. R. (2015) · Psychology of Addictive Behaviors · doi:10.1037/adb0000014 · Cited by 26
“…studies (e.g., Cunningham, Warme, Schaad, Wolf, & Leopold, 2007; Dickersin et al., 1992; Easterbrook et al., 1991), whereas studies receiving funding from pharmaceutical companies are less likely to be published compared with those not funded by pharmaceutical companies (Hall et al., 2007).”
BackgroundVerhagen, A. P., de Vet, H. C. W., Willemsen, S., & Stijnen, T. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.006 · Cited by 14
Metadata
- Authors
- Hall, R., de Antueno, C., & Webber, A.
- Venue
- Canadian Journal of Anesthesia
- Published
- 2007-05
- Volume
- 54
- Issue
- 5
- Pages
- 380-388
- ISSN
- 0832-610X (ISSN-L)
- 1496-8975
- OpenAlex
- W1603293637
- Cited by
- 54
- References
- 11
- FWCI
- 6.00
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.springer.com/tdm (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.