The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
In meta-analyses of proportion studies, funnel plots were found to be an inaccurate method of assessing publication bias
Abstract
Objective To assess the utility of funnel plots in assessing publication bias (PB) in meta-analyses of proportion studies. Study design and setting Meta-analysis simulation study and meta-analysis of published literature reporting peri-operative mortality after abdominal aortic aneurysm (AAA) repair. Data for the simulation study were stochastically generated. A literature search of Medline and Embase was performed to identify studies for inclusion in the published literature meta-analyses. Results The simulation study demonstrated that conventionally constructed funnel plots (log odds vs. 1/standard error [1/SE]) for extreme proportional outcomes were asymmetric despite no PB. Alternative funnel plots constructed using study size rather than 1/SE showed no asymmetry for extreme proportional outcomes. When used in meta-analyses of the mortality of AAA repair, these alternative funnel plots highlighted the possibility for conventional funnel plots to demonstrate asymmetry when there was no evidence of PB. Conclusion Conventional funnel plots used to assess for potential PB in meta-analyses are inaccurate for meta-analyses of proportion studies with low proportion outcomes. Funnel plots of study size against log odds may be a more accurate way of assessing for PB in these studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In addition, we did not assess the publication bias because funnel plots have been found to give erroneous results when pooling prevalence data and they are not recommended to be used when dealing with prevalence effect size (Hunter et al., 2014).”
From Autism in the context of joint hypermobility, hypermobility spectrum disorders, and Ehlers–Danlos syndromes: A systematic review and prevalence meta-analyses (2025) · cites this paper as Hunter et al., 2014
In the citing paper
“the usefulness of standard publication bias tests for proportional meta-analyses has been questioned, with funnel plots and statistical tests potentially yielding misleading results.66”
From Co-infections in people with COVID-19: a systematic review and meta-analysis. (2020) · cites this paper as [66]
Citations
Cites
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
Self-citationFunnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
Deeks, J. J., Macaskill, P., & Irwig, L. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.016 · Cited by 2,910
Comparison of two methods to detect publication bias in meta-analysis.
Peters, J. L. (2006) · JAMA · doi:10.1001/jama.295.6.676 · Cited by 2,278
The case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766
Dissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039
Self-citationFull publication of results initially presented in abstracts.
Scherer, R. W., Langenberg, P., & von Elm, E. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000005.pub3 · Cited by 997
Adjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2007) · Statistics in Medicine · doi:10.1002/sim.2889 · Cited by 716
Self-citationA meta-analysis of 50 years of ruptured abdominal aortic aneurysm repair
Bown, M. J., Sutton, A. J., BELL, P. R. F., & Sayers, R. D. (2002) · British journal of surgery · doi:10.1046/j.1365-2168.2002.02122.x · Cited by 561
Self-citationAsymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy
Song, F., Khan, K. S., Dinnes, J., & Sutton, A. J. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.88 · Cited by 485
Self-citationMisleading funnel plot for detection of bias in meta-analysis
Tang, J.-L., & Liu, J. L. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00204-8 · Cited by 420
Moreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-2 · Cited by 373
Self-citationMeta Analysis on Mortality of Ruptured Abdominal Aortic Aneurysms
Hoornweg, L. L., Storm-Versloot, M. N., Ubbink, D. T., Koelemay, M. J. W., Legemate, D. A., & Balm, R. (2008) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2007.11.019 · Cited by 239
Steyerberg, E. W. (1995) · Archives of Internal Medicine · doi:10.1001/archinte.155.18.1998 · Cited by 226
Juxtarenal aortic aneurysm repair
Jongkind, V., Yeung, K. K., Akkersdijk, G. J. M., Heidsieck, D. S. P., Reitsma, J. B., Tangelder, G. J., & Wisselink, W. (2010) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2010.01.049 · Cited by 204
Moreno, S. G., Sutton, A. J., Turner, E. H., Abrams, K. R., Cooper, N. J., Palmer, T. M., & Ades, A. E. (2009) · BMJ · doi:10.1136/bmj.b2981 · Cited by 184
Self-citationPerioperative Mortality of Elective Abdominal Aortic Aneurysm Surgery
Steyerberg, E. W. (1995) · Archives of Internal Medicine · doi:10.1001/archinte.1995.00430180108012 · Cited by 161
Endovascular repair of ruptured abdominal aortic aneurysms: A systematic review and meta-analysis
Mastracci, T. M., Garrido‐Olivares, L., Cinà, C. S., & Clase, C. M. (2008) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2007.07.052 · Cited by 149
Adriaensen, M. E. A. P. M., Bosch, J. L., Halpern, E. F., Hunink, M. G. M., & Gazelle, G. S. (2002) · Radiology · doi:10.1148/radiol.2243011675 · Cited by 137
Rayt, H. S., Sutton, A. J., London, N. J. M., Sayers, R. D., & Bown, M. J. (2008) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2008.08.008 · Cited by 131
Self-citationNordon, I. M., Karthikesalingam, A., Hinchliffe, R. J., Holt, P., Loftus, I. M., & THOMPSON, M. M. (2009) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2009.11.002 · Cited by 123
Systematic Review and Meta-analysis of 12 Years of Endovascular Abdominal Aortic Aneurysm Repair
Franks, S. C., Sutton, A. J., Bown, M. J., & Sayers, R. D. (2006) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2006.10.017 · Cited by 121
Self-citationElective surgery of abdominal aortic aneurysms in octogenarians: A systematic review
Henebiens, M., Vahl, A. C., & Koelemay, M. J. W. (2008) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2007.09.004 · Cited by 102
Sadat, U., Boyle, J. R., Walsh, S. R., Tang, T. Y., Varty, K., & Hayes, P. D. (2008) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2007.11.028 · Cited by 99
Grootenboer, N., van Sambeek, M. R. H. M., Arends, L. R., Hendriks, J. M., Hunink, M. G. M., & Bosch, J. (2010) · British journal of surgery · doi:10.1002/bjs.7134 · Cited by 93
Moreno, S. G., Sutton, A. J., Thompson, J. R., Ades, A. E., Abrams, K. R., & Cooper, N. J. (2012) · Statistics in Medicine · doi:10.1002/sim.4488 · Cited by 90
Self-citationMortality After Endovascular Repair of Ruptured Abdominal Aortic Aneurysms
Karkos, C. D. (2009) · Archives of Surgery · doi:10.1001/archsurg.2009.132 · Cited by 62
Weighting bias in meta-analysis of binary outcomes
Tang, J.-L. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00237-7 · Cited by 31
Publication Bias in Epidemiological Studies
Siddiqi, N. (2011) · Central European Journal of Public Health · doi:10.21101/cejph.a3581 · Cited by 27
Sajid, M. S., Desai, M., Haider, Z., Baker, D., & Hamilton, G. (2008) · Asian Journal of Surgery · doi:10.1016/s1015-9584(08)60071-8 · Cited by 21
Endovascular Repair of Ruptured Abdominal Aortic Aneurysms: Systematic Literature Review
Azizzadeh, A., Villa, M. A., Miller, C. C., Estrera, A. L., Coogan, S. M., & Safi, H. J. (2008) · Vascular · doi:10.2310/6670.2008.00039 · Cited by 19
Research works printed but not linked: 1
- Constantine, 2008, Publication bias, Vol. 1
Cited by
Sorted by citations
Munn, Z., Moola, S., Lisy, K., Riitano, D., & Tufanaru, C. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000054 · Cited by 3,131
Accelerating extinction risk from climate change
Urban, M. C. (2015) · Science · doi:10.1126/science.aaa4984 · Cited by 2,436
The prevalence and incidence of NAFLD worldwide: a systematic review and meta-analysis
Riazi, K., Azhari, H., Charette, J. H., Underwood, F. E., King, J. A., Afshar, E. E., Swain, M. G., Congly, S. E., Kaplan, G. G., & Shaheen, A.-A. (2022) · The Lancet Gastroenterology & Hepatology · doi:10.1016/s2468-1253(22)00165-0 · Cited by 2,246
Co-infections in people with COVID-19: a systematic review and meta-analysis.
Lansbury, L., Lim, B., Baskaran, V., & Lim, W. S. (2020) · Journal of Infection · doi:10.1016/j.jinf.2020.05.046 · Cited by 1,619
A new improved graphical and quantitative method for detecting bias in meta-analysis
Furuya-Kanamori, L., Barendregt, J. J., & Doi, S. A. R. (2018) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000141 · Cited by 1,160
Metadata
- Authors
- Hunter, J. P., Saratzis, A., Sutton, A. J., Boucher, R. H., Sayers, R. D., & Bown, M. J.
- Affiliations
- Leicester Royal Infirmary (GB)
- University Hospital Coventry (GB)
- University of Leicester (GB)
- Corresponding author
- Hunter, James P.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2014-08
- Volume
- 67
- Issue
- 8
- Pages
- 897-903
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2136145129
- Cited by
- 786
- References
- 32
- Citations by year
- 2026: 71 · 2025: 93 · 2024: 95 · 2023: 108 · 2022: 119 · 2021: 93 · 2020: 59 · 2019: 56 · 2018: 25 · 2017: 26 · 2016: 22 · 2015: 17 · 2014: 1
- FWCI
- 6.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute for Health and Care Research (CL-2014-11-001, MET-12-01-101)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, funnel plot, study size, abdominal aortic aneurysm repair, perioperative mortality, funnel plot asymmetry, simulation study
- MeSH terms
- Data Interpretation, Statistical, Humans, Stochastic Processes, Cardiovascular Surgical Procedures, Models, Statistical, Aortic Aneurysm, Abdominal, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
