CoChat
Sign Up
Cited by 1,039Meta-analysis and systematic reviewsPharmaceutical industry and healthcarescientometrics and bibliometrics research

Dissemination and publication of research findings: an updated review of related biases

Open access
Song, F, Parekh,, S, Hooper, L, Loke, YK, Ryder, J, Sutton, AJ, Hing, C, Kwok, CS +2 more·2010·Health Technology Assessment
DOI 10.3310/hta14080
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVES: To identify and appraise empirical studies on publication and related biases published since 1998; to assess methods to deal with publication and related biases; and to examine, in a random sample of published systematic reviews, measures taken to prevent, reduce and detect dissemination bias. DATA SOURCES: The main literature search, in August 2008, covered the Cochrane Methodology Register Database, MEDLINE, EMBASE, AMED and CINAHL. In May 2009, PubMed, PsycINFO and OpenSIGLE were also searched. Reference lists of retrieved studies were also examined. REVIEW METHODS: In Part I, studies were classified as evidence or method studies and data were extracted according to types of dissemination bias or methods for dealing with it. Evidence from empirical studies was summarised narratively. In Part II, 300 systematic reviews were randomly selected from MEDLINE and the methods used to deal with publication and related biases were assessed. RESULTS: Studies with significant or positive results were more likely to be published than those with non-significant or negative results, thereby confirming findings from a previous HTA report. There was convincing evidence that outcome reporting bias exists and has an impact on the pooled summary in systematic reviews. Studies with significant results tended to be published earlier than studies with non-significant results, and empirical evidence suggests that published studies tended to report a greater treatment effect than those from the grey literature. Exclusion of non-English-language studies appeared to result in a high risk of bias in some areas of research such as complementary and alternative medicine. In a few cases, publication and related biases had a potentially detrimental impact on patients or resource use. Publication bias can be prevented before a literature review (e.g. by prospective registration of trials), or detected during a literature review (e.g. by locating unpublished studies, funnel plot and related tests, sensitivity analysis modelling), or its impact can be minimised after a literature review (e.g. by confirmatory large-scale trials, updating the systematic review). The interpretation of funnel plot and related statistical tests, often used to assess publication bias, was often too simplistic and likely misleading. More sophisticated modelling methods have not been widely used. Compared with systematic reviews published in 1996, recent reviews of health-care interventions were more likely to locate and include non-English-language studies and grey literature or unpublished studies, and to test for publication bias. CONCLUSIONS: Dissemination of research findings is likely to be a biased process, although the actual impact of such bias depends on specific circumstances. The prospective registration of clinical trials and the endorsement of reporting guidelines may reduce research dissemination bias in clinical research. In systematic reviews, measures can be taken to minimise the impact of dissemination bias by systematically searching for and including relevant studies that are difficult to access. Statistical methods can be useful for sensitivity analyses. Further research is needed to develop methods for qualitatively assessing the risk of publication bias in systematic reviews, and to evaluate the effect of prospective registration of studies, open access policy and improved publication guidelines.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3310/hta14080

Cites

621 of 621 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • If authors fail to report adverse events or side effects in their abstracts, this will mislead the clinical decision of physicians and may even cause harm to patients.

    “If authors fail to report adverse events or side effects in their abstracts, this will mislead the clinical decision of the physicians and may even cause harm to patients”

    From Evaluation of reporting quality of abstracts of randomized controlled trials regarding patients with COVID-19 using the CONSORT statement for abstracts (2022) · cites this paper as Song et al., 2010

    The “Other evidence” section reports that published articles underreported adverse effects, and the “Clinical trials” section says publication bias could lead patients to receive a harmful treatment. Neither passage specifically addresses omissions from article abstracts or establishes that such omissions will mislead physicians’ clinical decisions; the paper describes patient harm as a possibility. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Although peer reviewers and journal editors can be biased in favour of positive findings, lack of publication seems primarily due to trial investigators or sponsors failing to submit negative or null results, rather than journals rejecting them.

    “Although peer reviewers and journal editors can be biased in favour of positive findings,119 lack of publication seems to be primarily due to trial investigators or sponsors failing to submit negative or null results, rather than journals rejecting them.114 120”

    From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [114]

  • Trials with statistically non-significant results are more prone to non-publication or long delay to publication (publication bias).

    “Despite providing important information relevant to patients, trials with statistically non-significant results are more prone to non-publication,114 or, when published, to a long delay to publication114 115 116—often referred to as publication bias.”

    From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [114]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Part of this paper’s reference list could not be split into single entries.

1 retracted

Flagged first, then by citations

Research works printed but not linked: 57

  1. Chan A-W, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004;291:2457-65.
  2. Chan A-W, Krleza-Jeri K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research [see comment]. Can Med Assoc J 2004;171:735-40.
  3. Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed [see comment]. J Clin Epidemiol 2005;58:882-93.
  4. Smith R. Editorial: What is publication? BMJ 1999;318:142.
  5. Edwards S, Liford RJ, Kiauka S. Different types of systematic review in health services research. In Black N, Brazier H, Fitzpatrick R, Reeves B, editors. Health services research methods: a guide to best practice. London: BMJ Books; 1998.
  6. Sohn D. Publications bias and the evaluation of psychotherapy efficacy in reviews of the research literature. Clin Psychol Rev 1996;16:147-56.
  7. Decullier E, Lheritier V, Chapuis F. Fate of biomedical research protocols and publication bias in France: retrospective cohort study [see comment]. BMJ 2005;331:19.
  8. Misakian AL, Bero LA. Publication bias and research on passive smoking: comparison of published and unpublished studies. JAMA 1998;280:250-3.
  9. Zimpel T, Windeler J. [Publications of dissertations on unconventional medical therapy and diagnosis procedures -a contribution to 'publication bias']. Forsch Komp Klas Nat 2000;7:71-4.
  10. Pich J, Carne X, Arnaiz J-A, Gomez B, Trilla A, Rodes J. Role of a research ethics committee in follow-up and publication of results [see comment]. Lancet 2003;361:1015-6.
  11. Loep M, Kleijnen J. Full publication of abstracts initially published in the Netherlands Journal of Medicine, 1999. Data obtained by personal communication with the author, Jo Kleijnen, Centre for Reviews and Dissemination, University of York, UK, June 1999.
  12. Delamere F, Williams H. To what extent are conference abstracts reporting randomised controlled trials of skin diseases published subsequently? [abstract]. XIII Cochrane Colloquium, Oct 22-26 2005; Melbourne, Australia.
  13. Eloubeidi MA, Wade SB, Provenzale D. Factors associated with acceptance and full publication of GI endoscopic research originally published in abstract form [see comment]. Gastrointest Endosc 2001;53:275-82.
  14. Kiroff GK. Publication bias in presentations to the Annual Scientific Congress. Aust NZ J Surg 2001;71:167-71.
  15. Klassen TP, Wiebe N, Russell K, Stevens K, Hartling L, Craig WR, et al. Abstracts of randomized controlled trials presented at the society for pediatric research meeting: an example of publication bias [comment]. Arch Pediat Adol Med 2002;156:474-9.
  16. Zaretsky Y, Imrie K. The fate of phase III trial abstracts presented at the American Society of Hematology [abstract]. Blood 2002;100:185a.
  17. Phillips CV. Publication bias in situ. BMC Med Res Methodol 2004;4:20.
  18. West RRJ, D.A. Publication bias in statistical overview of trials: example of psychological rehabilitation following myocardial infarction [abstract]. The 2nd International Conference on the Scientific Basis of Health Services and 5th Cochrane Colloquium, 8-12 October 1997, Amsterdam.
  19. Kyzas PA, Loizou KT, Ioannidis JPA. Selective reporting biases in cancer prognostic factor studies [see comment]. J Natl Cancer I 2005;97:1043-55.
  20. Dahlof CG, Pascual J, Dodick DW, Dowson AJ. Efficacy, speed of action and tolerability of almotriptan in the acute treatment of migraine: pooled individual patient data from four randomized, double-blind, placebo- controlled clinical trials. Cephalalgia 2006;26:400-8.
  21. Dibra A, Kastrati A, Alfonso F, Seyfarth M, Perez- Vizcayno MJ, Mehilli J, et al. Effectiveness of drug-eluting stents in patients with bare-metal in-stent restenosis: meta-analysis of randomized trials. J Am Coll Cardiol 2007;49:616-23.
  22. Lyman GH, Glaspy J. Are there clinical benefits with early erythropoietic intervention for chemotherapy- induced anemia? A systematic review. Cancer 2006;106:223-33.
  23. Anderson MA, Levsen J, Dusio ME, Bryant PJ, Brown SM, Burr CM, et al. Evidence-based factors in readmission of patients with heart failure. J Nurs Care Qual 2006;21:160-7.
  24. Asia PCSC. Coronary risk prediction for those with and without diabetes. Eur J Cardiovasc Prev Rehabil 2006;13:30-6.
  25. Brimble KS, Walker M, Margetts PJ, Kundhal KK, Rabbat CG. Meta-analysis: peritoneal membrane transport, mortality, and technique failure in peritoneal dialysis. J Am Soc Nephrol 2006;17:2591-8.
  26. Ding EL, Song Y, Malik VS, Liu S. Sex differences of endogenous sex hormones and risk of type 2 diabetes: a systematic review and meta-analysis. JAMA 2006;295:1288-99.
  27. Jewett M, Rendon R, Dranitsaris G, Drachenberg D, Tanguay S, Donnelly B, et al. Does prolonging the time to renal cancer surgery affect long-term cancer control: a systematic review of the literature. Can J Urol 2006;13(Suppl. 3):54-61.
  28. Mehra R, Moore BA, Crothers K, Tetrault J, Fiellin DA. The association between marijuana smoking and lung cancer: a systematic review. Arch Intern Med 2006;166:1359-67.
  29. Paydarfar JA, Birkmeyer NJ. Complications in head and neck surgery: a meta-analysis of postlaryngectomy pharyngocutaneous fistula. Arch Otolaryngol Head Neck Surg 2006;132:67-72.
  30. Rhodes RE, Smith NE. Personality correlates of physical activity: a review and meta-analysis. Br J Sports Med 2006;40:958-65.
  31. Singh G, Wu O, Langhorne P, Madhok R. Risk of acute myocardial infarction with nonselective non-steroidal anti-inflammatory drugs: a meta-analysis. Arthritis Res Ther 2006;8:R153.
  32. Zumkeller N, Brenner H, Zwahlen M, Rothenbacher D. Helicobacter pylori infection and colorectal cancer risk: a meta-analysis. Helicobacter 2006;11:75-80. Diagnostic reviews (n = 50)
  33. Carnes D, Ashby D, Underwood M. A systematic review of pain drawing literature: should pain drawings be used for psychologic screening? Clin J Pain 2006;22:449-57.
  34. de Graaf I, Prak A, Bierma-Zeinstra S, Thomas S, Peul W, Koes B. Diagnosis of lumbar spinal stenosis: a systematic review of the accuracy of diagnostic tests. Spine 2006;31:1168-76.
  35. Gisbert JP, de la Morena F, Abraira V. Accuracy of monoclonal stool antigen test for the diagnosis of H. pylori infection: a systematic review and meta-analysis. Am J Gastroenterol 2006;101:1921-30.
  36. Riquelme A, Calvo M, Salech F, Valderrama S, Pattillo A, Arellano M, et al. Value of adenosine deaminase (ADA) in ascitic fluid for the diagnosis of tuberculous peritonitis: a meta-analysis. J Clin Gastroenterol 2006;40:705-10.
  37. Arias A, Feinn R, Kranzler HR. Association of an Asn40Asp (A118G) polymorphism in the mu-opioid receptor gene with substance dependence: a meta- analysis. Drug Alcohol Depend 2006;83:262-8.
  38. Camargo MC, Mera R, Correa P, Peek RM Jr, Fontham ET, Goodman KJ, et al. Interleukin-1beta and interleukin-1 receptor antagonist gene polymorphisms and gastric cancer: a meta-analysis. Cancer Epidemiol Biomarkers Prev 2006;15:1674-87.
  39. Gao J, Shan G, Sun B, Thompson PJ, Gao X. Association between polymorphism of tumour necrosis factor alpha-308 gene promoter and asthma: a meta-analysis. Thorax 2006;61:466-71.
  40. Huang X, Chen P, Kaufer DI, Troster AI, Poole C. Apolipoprotein E and dementia in Parkinson disease: a meta-analysis. Arch Neurol 2006;63:189-93.
  41. Li H, Tai BC. RNASEL gene polymorphisms and the risk of prostate cancer: a meta-analysis. Clin Cancer Res 2006;12:5713-9.
  42. Noso S, Ikegami H, Fujisawa T, Kawabata Y, Asano K, Hiromine Y, et al. Association of SUMO4, as a candidate gene for IDDM5, with susceptibility to type 1 diabetes in Asian populations. Ann N Y Acad Sci 2006;1079:41-6.
  43. Shirts BH, Wood J, Yolken RH, Nimgaonkar VL. Association study of IL10, IL1beta, and IL1RN and schizophrenia using tag SNPs from a comprehensive database: suggestive association with rs16944 at IL1beta. Schizophr Res 2006;88:235-44.
  44. Tonjes A, Scholz M, Loeffler M, Stumvoll M. Association of Pro12Ala polymorphism in peroxisome proliferator- activated receptor gamma with Pre-diabetic phenotypes: meta-analysis of 57 studies on nondiabetic individuals. Diabetes Care 2006;29:2489-97.
  45. Zintzaras E, Kitsios G, Stefanidis I. Endothelial NO synthase gene polymorphisms and hypertension: a meta- analysis. Hypertension 2006;48:700-10.
  46. Cruciani M, Lipsky BA, Mengoli C, de Lalla F. Are granulocyte colony-stimulating factors beneficial in treating diabetic foot infections?: A meta-analysis. Diabetes Care 2005;28:454-60.
  47. Kasper JS, Giovannucci E. A meta-analysis of diabetes mellitus and the risk of prostate cancer. Cancer Epidemiol Biomarkers Prev 2006;15:2056-62.
  48. Nowak AK, Stockler MR, Chow PK, Findlay M. Use of tamoxifen in advanced-stage hepatocellular carcinoma. A systematic review. Cancer 2005;103:1408-14.
  49. Owen CG, Shah A, Henshaw K, Smeeth L, Sheikh A. Topical treatments for seasonal allergic conjunctivitis: systematic review and meta-analysis of efficacy and effectiveness. Br J Gen Pract 2004;54:451-6.
  50. Ownby RL, Crocco E, Acevedo A, John V, Loewenstein D. Depression and risk for Alzheimer disease: systematic review, meta-analysis, and metaregression analysis. Arch Gen Psychiatry 2006;63:530-8.
  51. Wellman RJ, Sugarman DB, DiFranza JR, Winickoff JP. The extent to which tobacco marketing and tobacco use in films contribute to children's use of tobacco: a meta- analysis. Arch Pediatr Adolesc Med 2006;160:1285-96.
  52. Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7:1-76.
  53. Moher D, Pham B, Lawson ML, Klassen TP. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health Technology Assessment 2003;7:1-90.
  54. Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Jama 2004;291:2457-65.
  55. Chan AW,.Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ 2005;330:753.
  56. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of Two Methods to Detect Publication Bias in Meta- analysis. JAMA: The Journal of the American Medical Association 2006;295:676-80.
  57. Edwards S, Lilford RJ, Kiauka S. Different types of systematic review in health services research. In Black N, Brazier J, Fitzpatrick R, Reeves B, eds. Health services research methods: a guide to best practice, pp 255-9. London: BMJ Books, 1998. Rimonabant for the treatment of overweight and obese people.

Cited by

Sorted by citations

Related

Authors
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I.
Affiliations
  • University of East Anglia (GB)
  • Watford General Hospital (GB)
Corresponding author
Song, F
ORCID
Venue
Health Technology Assessment
Publisher
National Institute for Health and Care Research
Published
2010-02
Volume
14
Issue
8
Pages
iii, ix-xi, 1
ISSN
  • 1366-5278 (ISSN-L)
  • 2046-4924
DOI
10.3310/hta14080
OpenAlex
W1994742944
Cited by
1,039
References
610
Citations by year
2026: 69 · 2025: 61 · 2024: 59 · 2023: 61 · 2022: 62 · 2021: 70 · 2020: 90 · 2019: 61 · 2018: 61 · 2017: 64 · 2016: 70 · 2015: 83 · 2014: 84 · 2013: 59 · 2012: 43
FWCI
12.47
Type
journal article
Language
en
Open access
Licence
Locations
  • Health Technology Assessment (publishedVersion, other-oa)
  • PubMed (publishedVersion)
  • Research Explorer (The University of Manchester) (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Leicester Research Archive (University of Leicester) (submittedVersion)
  • UEA Digital Repository (University of East Anglia) (submittedVersion)
Funders
Health Technology Assessment programme (06/92/02)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Pharmacology › Pharmaceutical industry and healthcare
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
Keywords
publication bias, reporting bias, selective reporting, grey literature, clinical trial registration, funnel plot, sensitivity analysis, reporting guidelines, complementary and alternative medicine, research integrity
MeSH terms
Humans, Review Literature as Topic, Bias, Publication Bias, Evidence-Based Medicine, Information Dissemination, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review