Cited by 789Meta-analysis and systematic reviewsStatistical Methods in Epidemiologyscientometrics and bibliometrics research

In meta-analyses of proportion studies, funnel plots were found to be an inaccurate method of assessing publication bias

Hunter, James P., Saratzis, Athanasios, Sutton, Alex J., Boucher, Rebecca H., Sayers, Robert D., Bown, Matthew J.·2014·Journal of Clinical Epidemiology

Abstract

Objective To assess the utility of funnel plots in assessing publication bias (PB) in meta-analyses of proportion studies. Study design and setting Meta-analysis simulation study and meta-analysis of published literature reporting peri-operative mortality after abdominal aortic aneurysm (AAA) repair. Data for the simulation study were stochastically generated. A literature search of Medline and Embase was performed to identify studies for inclusion in the published literature meta-analyses. Results The simulation study demonstrated that conventionally constructed funnel plots (log odds vs. 1/standard error [1/SE]) for extreme proportional outcomes were asymmetric despite no PB. Alternative funnel plots constructed using study size rather than 1/SE showed no asymmetry for extreme proportional outcomes. When used in meta-analyses of the mortality of AAA repair, these alternative funnel plots highlighted the possibility for conventional funnel plots to demonstrate asymmetry when there was no evidence of PB. Conclusion Conventional funnel plots used to assess for potential PB in meta-analyses are inaccurate for meta-analyses of proportion studies with low proportion outcomes. Funnel plots of study size against log odds may be a more accurate way of assessing for PB in these studies.

Integrity signals

Registry

Crossref · 10.1016/j.jclinepi.2014.03.003

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The authors did not assess publication bias because funnel plots have been found to give erroneous results when pooling prevalence data and are not recommended for prevalence effect size.

    “In addition, we did not assess the publication bias because funnel plots have been found to give erroneous results when pooling prevalence data and they are not recommended to be used when dealing with prevalence effect size (Hunter et al., 2014).”
  • The usefulness of standard publication bias tests for proportional meta-analyses has been questioned, with funnel plots and statistical tests potentially yielding misleading results.

    “the usefulness of standard publication bias tests for proportional meta-analyses has been questioned, with funnel plots and statistical tests potentially yielding misleading results.66”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Hunter, J. P., Saratzis, A., Sutton, A. J., Boucher, R. H., Sayers, R. D., & Bown, M. J.
Affiliations
  • Leicester Royal Infirmary (GB)
  • University Hospital Coventry (GB)
  • University of Leicester (GB)
Corresponding author
Hunter, James P.
Venue
Journal of Clinical Epidemiology
Publisher
Elsevier BV
Published
2014-08
Volume
67
Issue
8
Pages
897-903
ISSN
0895-4356 (ISSN-L)
OpenAlex
W2136145129
Cited by
786
References
32
Citations by year
2026: 71 · 2025: 93 · 2024: 95 · 2023: 108 · 2022: 119 · 2021: 93 · 2020: 59 · 2019: 56 · 2018: 25 · 2017: 26 · 2016: 22 · 2015: 17 · 2014: 1
FWCI
6.00
Type
journal article
Language
en
Open access
closed
Locations
  • Journal of Clinical Epidemiology (publishedVersion)
  • PubMed (publishedVersion)
Funders
National Institute for Health and Care Research (CL-2014-11-001, MET-12-01-101)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
Keywords
publication bias, funnel plot, study size, abdominal aortic aneurysm repair, perioperative mortality, funnel plot asymmetry, simulation study
MeSH terms
Data Interpretation, Statistical, Humans, Stochastic Processes, Cardiovascular Surgical Procedures, Models, Statistical, Aortic Aneurysm, Abdominal, Publication Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review