Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 124 of the 124 references it cites were found in a registry or an index.

Estimating the Difference Between Published and Unpublished Effect Sizes: A Meta-Review

Polanin, Joshua R., Tanner-Smith, Emily E., Hennessy, Emily A. · 2016 · Review of Educational Research

Abstract

Practitioners and policymakers rely on meta-analyses to inform decision making around the allocation of resources to individuals and organizations. It is therefore paramount to consider the validity of these results. A well-documented threat to the validity of research synthesis results is the presence of publication bias, a phenomenon where studies with large and/or statistically significant effects, relative to studies with small or null effects, are more likely to be published. We investigated this phenomenon empirically by reviewing meta-analyses published in top-tier journals between 1986 and 2013 that quantified the difference between effect sizes from published and unpublished research. We reviewed 383 meta-analyses of which 81 had sufficient information to calculate an effect size. Results indicated that published studies yielded larger effect sizes than those from unpublished studies ([Formula: see text] = 0.18, 95% confidence interval [0.10, 0.25]). Moderator analyses revealed that the difference was larger in meta-analyses that included a wide range of unpublished literature. We conclude that intervention researchers require continued support to publish null findings and that meta-analyses should include unpublished studies to mitigate the potential bias from publication status.

Overview

Integrity signals

Registry

Crossref · 10.3102/0034654315582067

Cites

124 of 124 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 123 references

Research works printed but not linked: 12

  1. 10.1037/00332909.125.1.64 · doi:10.1037/00332909.125.1.64
  2. 10.1037//0033-2909.128.2.203 · DOI · Paper page
  3. 10.1037/0033-2909.104.2.226 · DOI · Paper page
  4. Hopewell S., 2005, Publication bias in meta-analysis: Prevention, assessment and adjustments, 48
  5. 10.1037/0003-066X.48.12.1181 · DOI · Paper page
  6. Lipsey M. W., 2001, Applied social research methods: Vol. 49. Practical meta-analysis
  7. 10.3102/00346543065002129 · doi:10.3102/00346543065002129 · Paper page
  8. 10.1037/0033-2909.124.1.22 · DOI · Paper page
  9. Sterne J. A. C., 2005, Publication bias in meta-analysis: Prevention, assessment and adjustments, 75
  10. Sutton A. J., 2009, The handbook of research synthesis and meta-analysis, 435
  11. *Wood W., 1994, Psychological Bulletin, 115, 323, 10.1037/0033-2909.115.3.323 · doi:10.1037/0033-2909.115.3.323 · Paper page
  12. 10.1037/0033-2909.109.3.371 · DOI · Paper page

Cited by

See all 236

Metadata

Authors
Polanin, J. R., Tanner-Smith, E. E., & Hennessy, E. A.
Affiliations
  • Vanderbilt University (US)
  • Vanderbilt University
Venue
Review of Educational Research
Published
2016-03
Volume
86
Issue
1
Pages
207-236
ISSN
  • 0034-6543 (ISSN-L)
  • 1935-1046
OpenAlex
W2126293099
Cited by
234
References
123
Citations by year
2026: 9 · 2025: 18 · 2024: 20 · 2023: 26 · 2022: 24 · 2021: 45 · 2020: 25 · 2019: 27 · 2018: 17 · 2017: 11 · 2016: 7
FWCI
11.23
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
Review of Educational Research (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords
publication bias, unpublished research, meta-analysis, negative results, research validity, publication status, evidence synthesis, intervention research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Estimating the Difference Between Published and Unpublished Effect Sizes: A Meta-Review. https://app.cochat.ai/papers/10.3102/0034654315582067
The paper
Polanin, J. R., Tanner-Smith, E. E., & Hennessy, E. A. (2016). Estimating the Difference Between Published and Unpublished Effect Sizes: A Meta-Review. Review of Educational Research. https://doi.org/10.3102/0034654315582067

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.