Graphical augmentations to the funnel plot assess the impact of additional evidence on a meta-analysis
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: We aim to illustrate the potential impact of a new study on a meta-analysis, which gives an indication of the robustness of the meta-analysis. STUDY DESIGN AND SETTING: A number of augmentations are proposed to one of the most widely used of graphical displays, the funnel plot. Namely, 1) statistical significance contours, which define regions of the funnel plot in which a new study would have to be located to change the statistical significance of the meta-analysis; and 2) heterogeneity contours, which show how a new study would affect the extent of heterogeneity in a given meta-analysis. Several other features are also described, and the use of multiple features simultaneously is considered. RESULTS: The statistical significance contours suggest that one additional study, no matter how large, may have a very limited impact on the statistical significance of a meta-analysis. The heterogeneity contours illustrate that one outlying study can increase the level of heterogeneity dramatically. CONCLUSION: The additional features of the funnel plot have applications including 1) informing sample size calculations for the design of future studies eligible for inclusion in the meta-analysis; and 2) informing the updating prioritization of a portfolio of meta-analyses such as those prepared by the Cochrane Collaboration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569
Self-citationPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Self-citationThe case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766
Sterne, J. A. C., & Harbord, R. M. (2004) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0400400204 · Cited by 708 · cited 2×
“Additionally, a pseudo confidence interval can be used to indicate the region within which we would expect 95% of studies to lie if the studies are all estimating the same underlying effect [10] (the dotted sloping lines in Fig.”
MethodThe Science of Reviewing Researcha
OXMAN, A. D., & GUYATT, G. H. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26342.x · Cited by 472
Winkley, K., Landau, S., Eisler, I., & Ismail, K. (2006) · BMJ · doi:10.1136/bmj.38874.652569.55 · Cited by 359 · cited 1×
“The feature has since been widely used in meta-analysis [11,12] and implemented as a default setting into meta-analysis macros available for statistical packages such as Stata and R [10,13].”
MethodSterne, J. A. C., Becker, B. J., & Egger, M. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch5 · Cited by 341
Graphical displays for meta-analysis: An overview with suggestions for practice
Anzures‐Cabrera, J., & Higgins, J. P. T. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.6 · Cited by 233
Self-citation
Cited by
Lim, S. S., Davies, M. J., Norman, R. J., & Moran, L. J. (2012) · Human Reproduction Update · doi:10.1093/humupd/dms030 · Cited by 939
Probiotics for the prevention of Clostridium difficile-associated diarrhea in adults and children
Goldenberg, J. Z., Ma, S. S., Saxton, J. D., Martzen, M. R., Vandvik, P. O., Thorlund, K., Guyatt, G. H., & Johnston, B. C. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006095.pub3 · Cited by 741
Machado, G. C., Maher, C. G., Ferreira, P. H., Pinheiro, M. B., Lin, C.-W. C., Day, R. O., McLachlan, A. J., & Ferreira, M. L. (2015) · BMJ · doi:10.1136/bmj.h1225 · Cited by 580
Probiotics for the prevention of Clostridium difficile-associated diarrhea in adults and children
Goldenberg, J. Z., Yap, C., Lytvyn, L., Lo, C. K.-F., Beardsley, J., Mertz, D., & Johnston, B. C. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006095.pub4 · Cited by 550
Shen, N. T., Maw, A., Tmanova, L. L., Pino, A., Ancy, K., Crawford, C. V., Simon, M. S., & Evans, A. T. (2017) · Gastroenterology · doi:10.1053/j.gastro.2017.02.003 · Cited by 273
Metadata
- Authors
- Langan, D., Higgins, J. P. T., Gregory, W., & Sutton, A. J.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2012-05
- Volume
- 65
- Issue
- 5
- Pages
- 511-519
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2162893032
- Cited by
- 89
- References
- 12
- FWCI
- 2.60
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Graphical augmentations to the funnel plot assess the impact of additional evidence on a meta-analysis. https://app.cochat.ai/papers/10.1016/j.jclinepi.2011.10.009
- The paper
- Langan, D., Higgins, J. P. T., Gregory, W., & Sutton, A. J. (2012). Graphical augmentations to the funnel plot assess the impact of additional evidence on a meta-analysis. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2011.10.009
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.