Publication Bias in Recent Meta-Analyses
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
INTRODUCTION: Positive results have a greater chance of being published and outcomes that are statistically significant have a greater chance of being fully reported. One consequence of research underreporting is that it may influence the sample of studies that is available for a meta-analysis. Smaller studies are often characterized by larger effects in published meta-analyses, which can be possibly explained by publication bias. We investigated the association between the statistical significance of the results and the probability of being included in recent meta-analyses. METHODS: For meta-analyses of clinical trials, we defined the relative risk as the ratio of the probability of including statistically significant results favoring the treatment to the probability of including other results. For meta-analyses of other studies, we defined the relative risk as the ratio of the probability of including biologically plausible statistically significant results to the probability of including other results. We applied a Bayesian selection model for meta-analyses that included at least 30 studies and were published in four major general medical journals (BMJ, JAMA, Lancet, and PLOS Medicine) between 2008 and 2012. RESULTS: We identified 49 meta-analyses. The estimate of the relative risk was greater than one in 42 meta-analyses, greater than two in 16 meta-analyses, greater than three in eight meta-analyses, and greater than five in four meta-analyses. In 10 out of 28 meta-analyses of clinical trials, there was strong evidence that statistically significant results favoring the treatment were more likely to be included. In 4 out of 19 meta-analyses of observational studies, there was strong evidence that plausible statistically significant outcomes had a higher probability of being included. CONCLUSIONS: Publication bias was present in a substantial proportion of large meta-analyses that were recently published in four major medical journals.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709 · cited 2×
“Publication bias is one of the possible explanations of this phenomenon[17].”
BackgroundGelman, A. (2006) · Bayesian Analysis · doi:10.1214/06-ba117a · Cited by 4,050
Norman, R. E., Byambaa, M., De, R., Butchart, A., Scott, J., & Vos, T. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001349 · Cited by 3,387
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Deeks, J. J., Macaskill, P., & Irwig, L. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.016 · Cited by 2,910 · cited 1×
“Although a meta-analysis is typically preceded by an investigation of the presence of publication bias, the standard detection methods are characterized by a low power[11,18–22].”
MethodSelective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282 · cited 1×
“As a result, widely used publication bias tests often have a low power[18,20,22].”
BackgroundA modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints
Harbord, R. M., Egger, M., & Sterne, J. A. C. (2006) · Statistics in Medicine · doi:10.1002/sim.2380 · Cited by 2,267 · cited 1×
“Widely used publication bias methods such as the trim and fill method[34,39], Egger’s test[32], rank correlation test[33], and their modifications [18,19,40–42] are based on funnel plot asymmetry.”
MethodThe case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766
Ronksley, P. E., Brien, S. E., Turner, B. J., Mukamal, K. J., & Ghali, W. A. (2011) · BMJ · doi:10.1136/bmj.d671 · Cited by 1,610
Cited by
The use of google trends in health care research: a systematic review
Nuti, S. V., Wayda, B., Ranasinghe, I., Wang, S., Dreyer, R. P., Chen, S. I., & Murugiah, K. (2014) · PLoS ONE · doi:10.1371/journal.pone.0109583 · Cited by 1,086
“[ 13,14] This publication bias also may be due to researchers retroactively constructing hypotheses about interesting findings after the results are known for a given Google Trends experiment, which can be fast and easy to conduct .”
BackgroundPetti, S., Glendor, U., & Andersson, L. (2018) · Dental Traumatology · doi:10.1111/edt.12389 · Cited by 698
Empirical Comparison of Publication Bias Tests in Meta-Analysis
Lin, L., Chu, H., Murad, M. H., Hong, C., Qu, Z., Cole, S. R., & Chen, Y. (2018) · Journal of General Internal Medicine · doi:10.1007/s11606-018-4425-7 · Cited by 464
On the scientific superiority of conceptual replications for scientific progress
Crandall, C. S., & Sherman, J. W. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.10.002 · Cited by 344
Casaló, L. V., & Escario, J.-J. (2018) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2017.11.237 · Cited by 302
Metadata
- Authors
- Kicinski, M.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-11-27
- Volume
- 8
- Issue
- 11
- Pages
- e81823
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2038606278
- Cited by
- 97
- References
- 28
- FWCI
- 12.55
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Publication Bias in Recent Meta-Analyses. https://app.cochat.ai/papers/10.1371/journal.pone.0081823
- The paper
- Kicinski, M. (2013). Publication Bias in Recent Meta-Analyses. PLoS ONE. https://doi.org/10.1371/journal.pone.0081823
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.