The DOI matches its Crossref record; 44 of the 45 references it cites were found in a registry or an index.
Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials
Open accessAbstract
Funnel plots, and tests for funnel plot asymmetry, have been widely used to examine bias in the results of meta-analyses. Funnel plot asymmetry should not be equated with publication bias, because it has a number of other possible causes. This article describes how to interpret funnel plot asymmetry, recommends appropriate tests, and explains the implications for choice of meta-analysis model
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“so the non-significant tendencies to asymmetry and small study effects identified by the Eggers and Begg tests, presumably caused by the similar SEM of most of the studies, can be assumed to be trustworthy [30].”
From Are Dietary Proteins the Key to Successful Body Weight Management? A Systematic Review and Meta-Analysis of Studies Assessing Body Weight Outcomes after Interventions with Increased Dietary Protein (2021) · cites this paper as [30]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“publication bias was not assessed in this review because there was no forest plot with at least ten studies (Sterne et al. 2011).”
From The Effect of Mindfulness-Based and Acceptance Commitment Therapy-Based Interventions to Improve the Mental Well-Being Among Parents of Children with Developmental Disabilities: A Systematic Review and Meta-Analysis. (2022) · cites this paper as Sterne et al. 2011
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Publication bias was assessed using funnel plot analysis and Egger's test where the number of studies for the outcome of interest was ≥10”
From IVF/ICSI or surgery as first approach for the treatment of infertility associated with ovarian and deep infiltrating endometriosis? A systematic review and meta-analysis. (2026) · cites this paper as Sterne et al., 2011
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For those comparisons in which data were obtained from at least ten studies (Sterne et al. 2011), publication bias was assessed by visual inspection of funnel plot asymmetry.”
From Analysis of sex-based differences in energy substrate utilization during moderate-intensity aerobic exercise (2022) · cites this paper as Sterne et al. 2011
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Instead, the asymmetry likely represents a ‘small study bias’, where methodological approaches inherent in small studies lead to stronger effects with greater variance (Peters et al., 2008; Sterne et al., 2011).”
From The Distributed Practice Effect on Classroom Learning: A Meta-Analytic Review of Applied Research (2025) · cites this paper as Sterne et al., 2011
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Publication bias was assessed by visual inspection of funnel plot and by Egger test to assess asymmetry, as well as by Begg test to assess small study effects [29,30,31].”
From Are Dietary Proteins the Key to Successful Body Weight Management? A Systematic Review and Meta-Analysis of Studies Assessing Body Weight Outcomes after Interventions with Increased Dietary Protein (2021) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If a meta-analysis of a medication adherence measure had adequate power (ie, ≥10 cohorts), funnel plots were generated and visually inspected, and the Egger test was used to assess for small study effects (funnel plot asymmetry) potentially caused by reporting bias.30”
From Adherence to PCSK9 Inhibitors in Clinical Practice (2026) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we only calculated bias statistics if the total number of correspondence audits for a given analysis equalled or exceeded ten, otherwise, the statistical power could be too low to detect asymmetry.”
From The state of hiring discrimination: A meta-analysis of (almost) all recent correspondence experiments (2023) · cites this paper as Sterne et al., 2011
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Because the symmetry of funnel plots could be influence by various factors, such as publication bias, and/or small-study effects, it was difficult to figure out the cause of asymmetry, especially when the sample size of each comparison was less than ten [82].”
From Efficacy and safety of curcuminoids alone in alleviating pain and dysfunction for knee osteoarthritis: a systematic review and meta-analysis of randomized controlled trials (2022) · cites this paper as [82]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To assess risk of publication bias, we plotted each trial's effect against the standard errors (Sterne et al., 2011), creating a single effect size per study per outcome.”
From Shared Picture Book Reading Interventions for Child Language Development: A Systematic Review and Meta-Analysis (2020) · cites this paper as Sterne et al., 2011
The cited source says what the claim attributes to it.
In the citing paper
“Notably, heterogeneity can also influence funnel plot asymmetry (Sterne et al., 2011).”
From The Distributed Practice Effect on Classroom Learning: A Meta-Analytic Review of Applied Research (2025) · cites this paper as Sterne et al., 2011
The cited source says what the claim attributes to it.
In the citing paper
“Consequently, the Egger test was not employed to formally assess funnel plot asymmetry, as it is not recommended for such small numbers of studies [25].”
The cited source says what the claim attributes to it.
In the citing paper
“We were not able to explore possible publication bias through a funnel plot (Sterne 2011), as we did not combine a sufficient number of trials.”
From Mobile technologies to support healthcare provider to healthcare provider communication and management of care (2020) · cites this paper as Sterne 2011
The cited source says what the claim attributes to it.
In the citing paper
“Funnel plots were generated for visual assessment for asymmetry and potential publication bias 31 (see Figures S20–S29).”
From Systematic review and meta-analysis of protein intake to support muscle mass and function in healthy adults (2022) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“Typically tests for funnel plot asymmetry should only be used where there are at least 10 studies in each meta-analysis (Sterne et al., 2011).”
From Nature-based outdoor activities for mental and physical health: Systematic review and meta-analysis (2021) · cites this paper as Sterne et al., 2011
The cited source says what the claim attributes to it.
In the citing paper
“Presence of publication bias was assessed with funnel plots, 22 Rosenthal's fail-safe N , 23 and Egger's regression intercept. 24”
From Light Therapy for Patients With Bipolar Depression: Systematic Review and Meta-Analysis of Randomized Controlled Trials (2020) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“To explore publication and small study biases, a funnel plot and Egger ' s test for asymmetry was used where at least 10 different-sized studies were pooled. 62”
From High-velocity power training has similar effects to traditional resistance training for functional performance in older adults: a systematic review (2023) · cites this paper as [62]
The cited source says what the claim attributes to it.
In the citing paper
“Small study effects (e.g. publication bias) were assessed when there were 10 or more studies in the meta-analysis by visually examining the funnel plot, supplemented by Egger’s test of funnel plot asymmetry [39, 40]. Duval and Tweedie’s trim and fill procedure was used to estimate the effect size after imputing potentially missing studies [41].”
From Systematic review and meta-analysis of Mental Health First Aid training: Effects on knowledge, stigma, and helping behaviour (2018) · cites this paper as [40]
The cited source says what the claim attributes to it.
In the citing paper
“this asymmetry could exist due to chance, as the analysis contained few studies with few participants (Sterne et al., 2011).”
From The effects of physical activity on cortisol and sleep: A systematic review and meta-analysis. (2022) · cites this paper as Sterne et al., 2011
The cited source says what the claim attributes to it.
In the citing paper
“However, for each risk factor, no one included more than ten studies; these numbers were too small so that the test efficacy of funnel plots and egger's test was insufficient [31]. Therefore, funnel plots and Egger's test were not assessed to calculate the risk of bias.”
From Preoperative Risk Factors for Short-Term Postoperative Mortality of Acute Mesenteric Ischemia after Laparotomy: A Systematic Review and Meta-Analysis (2020) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“We assessed the latter for evidence of asymmetry, and therefore possible publication bias or other small study effects, using the Egger test, 35 if there were suf fi cient ( 10) eligible studies included in the meta-analysis, in line with recommendations. 36”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [36]
The cited source says what the claim attributes to it.
In the citing paper
“According to Sterne et al. (2011) it is not recommended to use tests for funnel plot asymmetry when the number of studies included in the meta-analysis is <10 as statistical power is usually too low to distinguish chance from real asymmetry.”
From The effect of internet-delivered cognitive behavioral therapy for depression and anxiety on quality of life: A meta-analysis of randomized controlled trials (2023) · cites this paper as Sterne et al., 2011
The cited source says what the claim attributes to it.
In the citing paper
“However, not all asymmetric funnel plots are due to issues that impact meta-analytic inference (Sterne et al., 2011).”
From Can MRI measure myelin? Systematic review, qualitative assessment, and meta-analysis of studies validating microstructural imaging with myelin histology (2021) · cites this paper as Sterne et al., 2011
The cited source says what the claim attributes to it.
In the citing paper
“Due to the small number of included studies, publication bias was not formally assessed using funnel plots or asymmetry tests, as these methods have low statistical power under such conditions (Sterne et al., 2011).”
From Efficacy and safety of small interfering RNA (siRNA) therapies for hypertriglyceridemia and mixed dyslipidemia: an updated systematic review and meta-analysis (2026) · cites this paper as Sterne et al., 2011
The cited source says what the claim attributes to it.
In the citing paper
“Because the test for a publication is not advised for fewer than 10 studies, Egger’s test was not performed.20”
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“To assess risk of bias, the Cochrane Collaboration's risk-of-bias tool (Rob2) was used (Sterne et al., 2011).”
From Effects of voluntary slow breathing on heart rate and heart rate variability: A systematic review and a meta-analysis (2022) · cites this paper as Sterne et al., 2011
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Included studies exceeded the minimum number of effect sizes to be pooled [98].”
From Estimating the Effect of Aerobic Exercise Training on Novel Lipid Biomarkers: A Systematic Review and Multivariate Meta-Analysis of Randomized Controlled Trials (2023) · cites this paper as [98]
Experimental — accuracy not yet measuredIn the citing paper
“This may be due to several reasons, including true heterogeneity (i.e., smaller studies differ from larger ones in terms of study population, exposure levels, etc.), selection bias (i.e., the selective publication of small studies showing an effect), bias in design or analysis, or chance [77,78].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [77]
In the citing paper
“Publication bias was examined by inspecting the funnel plot [44] and conducting the Egger test of the intercept with a one-tailed significance level of α=.05 [45]. In addition, we used Duval and Tweedie’s trim and fill procedure [46] to adjust the effect size for missing studies.”
In the citing paper
“Based on current recommendations, at least 10 studies are needed to examine reporting bias using funnel plots and, if the heterogeneity is high, the minimum number of studies may be substantially more than 10. 34”
From Global prevalence of sarcopenia and severe sarcopenia: a systematic review and meta‐analysis (2022) · cites this paper as [34]
In the citing paper
“Trialists may have implemented interventions less thoroughly in larger trials, thus explaining the more positive results in smaller trials 57 .”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [57]
In the citing paper
“Small trials generally are conducted before larger trials are established, and in the intervening years standard, control treatments may have improved, which can reduce the relative efficacy of the experimental treatment.”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [57]
In the citing paper
“Brief contact interventions reduce the number of suicide attempts per person but not the total number of suicide attempts or suicide deaths.9”
In the citing paper
“We constructed funnel plots for only 1 analysis due to the small number of studies included in each analysis (n<10) [63,64].”
From Tailored web-based interventions for pain: Systematic review and meta-analysis (2017) · cites this paper as [64]
In the citing paper
“Several reviews could not evaluate the presence of publication bias because of the small number of studies in the meta-analyses”
From Association between the reproductive health of young women and cardiovascular disease in later life: umbrella review (2020) · cites this paper as [123]
In the citing paper
“The Begg’s and Egger’s tests were not used because there were not more than 10 subjects in each group.15”
From A comprehensive analysis of the efficacy and safety of COVID-19 vaccines (2021) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Self-citationThe file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
Self-citationDeeks, J. J., Macaskill, P., & Irwig, L. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.016 · Cited by 2,910
Self-citationWood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538
Self-citationSelective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Self-citationComparison of two methods to detect publication bias in meta-analysis.
Peters, J. L. (2006) · JAMA · doi:10.1001/jama.295.6.676 · Cited by 2,278
Self-citationA modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints
Harbord, R. M., Egger, M., & Sterne, J. A. C. (2006) · Statistics in Medicine · doi:10.1002/sim.2380 · Cited by 2,267
Self-citation
Flagged first, then by citations
1 of the 45 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Items 1,001–1,100 of 1,118
Masi, A., Lampit, A., Glozier, N., Hickie, I. B., & Guastella, A. J. (2015) · Translational Psychiatry · doi:10.1038/tp.2015.143 · Cited by 78
Rupp, C., Doebler, P., Ehring, T., & Vossbeck‐Elsebusch, A. N. (2017) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2039 · Cited by 76
Launbo, N., Zobel, E. H., von Scholten, B. J., Færch, K., Jørgensen, P. G., & Christensen, R. H. (2021) · Obesity Reviews · doi:10.1111/obr.13136 · Cited by 76
Domhardt, M., Letsch, J., Kybelka, J., Koenigbauer, J., Doebler, P., & Baumeister, H. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.06.059 · Cited by 75
Gesel, S. A., LeJeune, L. M., Chow, J. C., Sinclair, A. C., & Lemons, C. J. (2021) · Journal of Learning Disabilities · doi:10.1177/0022219420970196 · Cited by 83
Young, I., Waddell, L., Harding, S., Greig, J., Mascarenhas, M., Sivaramalingam, B., Pham, M. T., & Papadopoulos, A. (2015) · BMC Public Health · doi:10.1186/s12889-015-2171-x · Cited by 83
Lam, R. W., Teng, M. Y., Jung, Y.-E., Evans, V. C., Gottlieb, J. F., Chakrabarty, T., Michalak, E. E., Murphy, J. K., Yatham, L. N., & Sit, D. K. (2020) · The Canadian Journal of Psychiatry · doi:10.1177/0706743719892471 · Cited by 70
Qin, W., Yang, J., Deng, C., Ruan, Q., & Duan, K. (2024) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.15386 · Cited by 72
Madsen, K. S., Chi, Y., Metzendorf, M.-I., Richter, B., & Hemmingsen, B. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008558.pub2 · Cited by 70
Long-term effects of weight-reducing drugs in people with hypertension
Siebenhofer, A., Winterholer, S., Jeitler, K., Horvath, K., Berghold, A., Krenn, C., & Semlitsch, T. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007654.pub5 · Cited by 70
Vollbehr, N. K., Bartels-Velthuis, A. A., Nauta, M. H., Castelein, S., Steenhuis, L. A., Hoenders, H. J. R., & Ostafin, B. D. (2018) · PLOS ONE · doi:10.1371/journal.pone.0204925 · Cited by 69
Chua, J. Y. X., Tam, W., & Shorey, S. (2020) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13164 · Cited by 69
Lauzier, F., Turgeon, A. F., Boutin, A., Shemilt, M., Côté, I., Lachance, O., Archambault, P. M., Lamontagne, F., Moore, L., Bernard, F., et al. (2014) · Critical Care Medicine · doi:10.1097/ccm.0000000000000046 · Cited by 68
Lau, Y., Cheng, J.-Y., Wong, S.-H., Yen, K.-Y., & Cheng, L.-J. (2021) · World Journal of Psychiatry · doi:10.5498/wjp.v11.i4.133 · Cited by 68
Werling, A. M., Kuzhippallil, S., Emery, S., Walitza, S., & Drechsler, R. (2022) · Journal of Behavioral Addictions · doi:10.1556/2006.2022.00007 · Cited by 68
Hoppen, T. H., Jehn, M., Holling, H., Mutz, J., Kip, A., & Morina, N. (2023) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000809 · Cited by 69
Bobos, P., Nazari, G., Szekeres, M., Lalone, E. A., Ferreira, L., & MacDermid, J. C. (2019) · Journal of Hand Therapy · doi:10.1016/j.jht.2018.09.012 · Cited by 64
Luo, X., Chen, Y., Chen, J., Zhang, Y., Li, M., Xiong, C., & Yan, J. (2022) · Supportive Care in Cancer · doi:10.1007/s00520-021-06568-0 · Cited by 57
Sebastiá-Rico, J., Soriano, J. M., González-Gálvez, N., & Martínez-Sanz, J. M. (2023) · Nutrients · doi:10.3390/nu15051160 · Cited by 66
Park, H. Y., Suh, C. H., Woo, S., Kim, P. H., & Kim, K. W. (2022) · Korean Journal of Radiology · doi:10.3348/kjr.2021.0808 · Cited by 61
Song, H., Chan, J. S., & Ryan, C. (2024) · Journal of Cross-Cultural Psychology · doi:10.1177/00220221241285006 · Cited by 62
“Publication bias was checked for strategies where the studies ≥ 10 (Sterne et al., 2011), using the Begg funnel plots and Egger’s regression test (Egger et al., 1997).”
Cognitive training for post-acute traumatic brain injury: a systematic review and meta-analysis
Hallock, H., Collins, D., Lampit, A., Deol, K., Fleming, J., & Valenzuela, M. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00537 · Cited by 60
Post-Traumatic Stress Disorder After Traumatic Brain Injury—A Systematic Review and Meta-Analysis
Iljazi, A., Ashina, H., Al-Khazali, H. M., Lipton, R. B., Ashina, M., Schytz, H. W., & Ashina, S. (2020) · Neurological Sciences · doi:10.1007/s10072-020-04458-7 · Cited by 62
Huang, G., Chu, H., Chen, R., Liu, D., Banda, K. J., O’Brien, A. P., Jen, H.-J., Chiang, K.-J., Chiou, J.-F., & Chou, K.-R. (2022) · Journal of Global Health · doi:10.7189/jogh.12.05028 · Cited by 64
Martínez-González, N. A., Rosemann, T., Djalali, S., Huber-Geismann, F., & Tandjung, R. (2015) · Medical Care Research and Review · doi:10.1177/1077558715586297 · Cited by 60
Acosta‐Manzano, P., Rodriguez‐Ayllon, M., Acosta, F. M., Niederseer, D., & Niebauer, J. (2020) · Obesity Reviews · doi:10.1111/obr.13007 · Cited by 59
Rothrock, S. G., Cassidy, D. D., Barneck, M., Schinkel, M., Guetschow, B., Myburgh, C., Nguyen, L., Earwood, R., Nanayakkara, P. W. B., Nannan Panday, R. S., et al. (2020) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2020.04.042 · Cited by 57
He, X.-K., Su, T.-T., Si, J.-M., & Sun, L.-M. (2016) · Medicine · doi:10.1097/md.0000000000002749 · Cited by 59
School Start Times, Sleep, and Youth Outcomes: A Meta-analysis
Yip, T., Wang, Y., Xie, M., Ip, P. S., Fowle, J., & Buckhalt, J. (2022) · Pediatrics · doi:10.1542/peds.2021-054068 · Cited by 58
Ong, C. W., Barthel, A. L., & Hofmann, S. G. (2024) · Behavior Therapy · doi:10.1016/j.beth.2023.05.007 · Cited by 61
Moelands, S. V., Lucassen, P. L., Akkermans, R. P., De Grauw, W. J., & Van de Laar, F. A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005061.pub3 · Cited by 57
Chromium picolinate supplementation for overweight or obese adults
Tian, H., Guo, X., Wang, X., He, Z., Sun, R., Ge, S., & Zhang, Z. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010063.pub2 · Cited by 57
Psychological interventions for acute pain after open heart surgery
Ziehm, S., Rosendahl, J., Barth, J., Strauss, B. M., Mehnert, A., & Koranyi, S. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009984.pub3 · Cited by 56
Effectiveness of multidisciplinary programmes for clinical pain conditions: an umbrella review.
Dragioti, E., Evangelou, E., Larsson, B., & Gerdle, B. (2018) · Journal of Rehabilitation Medicine · doi:10.2340/16501977-2377 · Cited by 53
Qin, W., Yang, J., Ni, Y., Deng, C., Ruan, Q., Ruan, J., Zhou, P., & Duan, K. (2024) · Endocrine · doi:10.1007/s12020-024-03896-z · Cited by 55
“Consequently, the Egger test was not employed to formally assess funnel plot asymmetry, as it is not recommended for such small numbers of studies [25].”
MethodHamel, C., Lang, E., Morissette, K., Beck, A., Stevens, A., Skidmore, B., Colquhoun, H., LeBlanc, J., Moore, A., Riva, J. J., et al. (2019) · Systematic Reviews · doi:10.1186/s13643-018-0930-3 · Cited by 55
Self-citationKaripidis, K., Baaken, D., Loney, T., Blettner, M., Brzozek, C., Elwood, M., Narh, C., Orsini, N., Röösli, M., Paulo, M. S., et al. (2024) · Environment International · doi:10.1016/j.envint.2024.108983 · Cited by 58
Benchmarks of Significant Change After Aphasia Rehabilitation
Gilmore, N., Dwyer, M., & Kiran, S. (2019) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2018.08.177 · Cited by 54
Animal-assisted Psychotherapy and Trauma: A Meta-analysis
Germain, S. M., Wilkie, K. D., Milbourne, V. M. K., & Theule, J. (2018) · Anthrozoös · doi:10.1080/08927936.2018.1434044 · Cited by 49
Meta‐analysis of clinical outcomes of PCSK9 modulators in patients with established ASCVD
Talasaz, A. H., Ho, A. (., Bhatty, F., Koenig, R. A., Dixon, D. L., Baker, W. L., & Van Tassell, B. W. (2021) · Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy · doi:10.1002/phar.2635 · Cited by 48
Mueller, K. F., Meerpohl, J. J., Briel, M., Antes, G., von Elm, E., Lang, B., Motschall, E., Schwarzer, G., & Bassler, D. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.015 · Cited by 48
Self-citationTeja, B., Berube, M., Pereira, T. V., Law, A. C., Schanock, C., Pang, B., Wunsch, H., Walkey, A. J., & Bosch, N. A. (2024) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202310-1785oc · Cited by 48
Dworschak, C., Heim, E., & Maercker, A. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100498 · Cited by 48
Kaw, R., Pasupuleti, V., Wayne Overby, D., Deshpande, A., Coleman, C. I., Ioannidis, J. P. A., & Hernandez, A. V. (2014) · Surgery for Obesity and Related Diseases · doi:10.1016/j.soard.2014.04.008 · Cited by 47
Self-citationNadinda, P. G., van Ryckeghem, D. M. L., & Peters, M. L. (2022) · Pain · doi:10.1097/j.pain.0000000000002521 · Cited by 51
Solmi, M., Correll, C. U., Carvalho, A. F., & Ioannidis, J. P. A. (2018) · Epidemiology and Psychiatric Sciences · doi:10.1017/s204579601800032x · Cited by 44
Self-citationDiagnostic biomarkers for obsessive-compulsive disorder: A reasonable quest or ignis fatuus?
Fullana, M. A., Abramovitch, A., Via, E., López-Sola, C., Goldberg, X., Reina, N., Fortea, L., Solanes, A., Buckley, M. J., Ramella-Cravaro, V., et al. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.08.008 · Cited by 44
Meta-analysis of sleep deprivation in the acute treatment of bipolar depression
Gottlieb, J. F., Goel, N., Chen, S., & Young, M. A. (2021) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13255 · Cited by 44
A Meta-Regression of Racial Disparities in Wellbeing Outcomes During and After Foster Care
Kennedy, R. S., Potter, M. H., & Font, S. A. (2023) · Trauma, Violence, & Abuse · doi:10.1177/15248380221111481 · Cited by 44
Chua, J. Y. X., & Shorey, S. (2022) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-021-04893-1 · Cited by 44
“As a minimum of ten studies in a forest plot was recommended to produce a meaningful funnel plot to analyze publication bias, publication bias was not assessed in this review because there was no forest plot with at least ten studies (Sterne et al. 2011).”
BackgroundLaranjeira, F. O., de Andrade, K. R. C., Figueiredo, A. C. M. G., Silva, E. N., & Pereira, M. G. (2018) · PLOS ONE · doi:10.1371/journal.pone.0194801 · Cited by 41
Brini, S., Brudasca, N. I., Hodkinson, A., Kaluzinska, K., Wach, A., Storman, D., Prokop-Dorner, A., Jemioło, P., & Bala, M. M. (2023) · Clinical Psychology Review · doi:10.1016/j.cpr.2022.102236 · Cited by 44
Amini-Salehi, E., Hassanipour, S., Keivanlou, M.-H., Shahdkar, M., Orang Goorabzarmakhi, M., Vakilpour, A., Joukar, F., Hashemi, M., Sattari, N., Javid, M., et al. (2024) · Nutrition Reviews · doi:10.1093/nutrit/nuad086 · Cited by 42
Bürkner, P.-C., Bittner, N., Holling, H., & Buhlmann, U. (2017) · PLoS ONE · doi:10.1371/journal.pone.0173660 · Cited by 41
Karran, E. L., Grant, A. R., Lee, H., Kamper, S. J., Williams, C. M., Wiles, L. K., Shala, R., Poddar, C. V., Astill, T., & Moseley, G. L. (2023) · BMC Public Health · doi:10.1186/s12889-023-15329-z · Cited by 39
Doi, S. A. R., Furuya‐Kanamori, L., Toft, E., Musa, O. A. H., Islam, N., Clark, J., & Thalib, L. (2020) · Obesity Reviews · doi:10.1111/obr.12964 · Cited by 39
Li, S. N., Chien, W. T., Lam, S. K. K., Chen, Z. Y., & Ma, X. (2024) · Research in Autism Spectrum Disorders · doi:10.1016/j.rasd.2024.102389 · Cited by 40
Intermittent fasting for the prevention of cardiovascular disease
Allaf, M., Elghazaly, H., Mohamed, O. G., Fareen, M. F. K., Zaman, S., Salmasi, A.-M., Tsilidis, K., & Dehghan, A. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013496.pub2 · Cited by 98
Adjunctive folate for major mental disorders: A systematic review
Zheng, W., Li, W., Qi, H., Xiao, L., Sim, K., Ungvari, G. S., Lu, X.-B., Huang, X., Ning, Y.-P., & Xiang, Y.-T. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.01.096 · Cited by 37
Naghipour, A., Amini-Salehi, E., Orang Gorabzarmakhi, M., Shahdkar, M., Fouladi, B., Alipourfard, I., & Sanat, Z. M. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02299-x · Cited by 37
“The publication bias was assessed by visual inspection of the funnel plot and Egger regression test, and P value 0.1 was determined as the level of significance [41, 42].”
MethodInfluentialHill Almeida, L. M., Flicker, L., Hankey, G. J., Golledge, J., Yeap, B. B., & Almeida, O. P. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.04.133 · Cited by 38
Mahapatro, A., Bawna, F., Kumar, V., Daryagasht, A. A., Gupta, S., Raghuma, N., Moghdam, S. S., Kolla, A., Mahapatra, S. S., Sattari, N., et al. (2023) · Clinical Nutrition ESPEN · doi:10.1016/j.clnesp.2023.07.087 · Cited by 37
el Hadouchi, M., Kiers, H., de Vries, R., Veenhof, C., & van Dieën, J. (2022) · European Review of Aging and Physical Activity · doi:10.1186/s11556-022-00297-x · Cited by 35
Odgers, K., Kershaw, K. A., Li, S. H., & Graham, B. M. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104203 · Cited by 35
Systematic review of self-monitoring of blood glucose in patients with type 2 diabetes
Chircop, J., Sheffield, D., & Kotera, Y. (2021) · Nursing Research · doi:10.1097/nnr.0000000000000542 · Cited by 34
Lo, W.-C., Ou, S.-H., Chou, C.-L., Chen, J.-S., Wu, M.-Y., & Wu, M.-S. (2021) · Journal of Nephrology · doi:10.1007/s40620-020-00957-0 · Cited by 34
Victorson, D. E., Sauer, C. M., Wolters, L., Maletich, C., Lukoff, K., & Sufrin, N. (2020) · Mindfulness · doi:10.1007/s12671-020-01373-y · Cited by 33
Stefanou, M.-I., Palaiodimou, L., Theodorou, A., Safouris, A., Fischer, U., Kelly, P. J., Dawson, J., Katan, M., Katsanos, A. H., Lambadiari, V., et al. (2024) · Therapeutic Advances in Neurological Disorders · doi:10.1177/17562864241281903 · Cited by 36
Feng, J., Li, Z., Tian, L., Mu, P., Hu, Y., Xiong, F., & Ma, X. (2022) · BMC Complementary Medicine and Therapies · doi:10.1186/s12906-022-03740-9 · Cited by 33
Teraz, K., Šlosar, L., Paravlić, A. H., de Bruin, E. D., & Marusic, U. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.837710 · Cited by 32
Lan, X.-J., Yang, X.-H., Mo, Y., Deng, C.-J., Huang, X.-B., Cai, D.-B., & Zheng, W. (2024) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2024.104032 · Cited by 32
Bauer, K., Lau, T., Schwille‐Kiuntke, J., Schild, S., Hauner, H., Stengel, A., Zipfel, S., & Mack, I. (2020) · European Eating Disorders Review · doi:10.1002/erv.2741 · Cited by 31
“The funnel plot depicted in Data S4 shows an asymmetry with a deficiency in the lower corner on the right and may implicate a reporting bias (Sterne et al., 2011).”
BackgroundTao, S., Geng, Y., Li, M., Ye, J., & Liu, Z. (2022) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2022.111071 · Cited by 34
Leeyaphan, C., Ong, J. J., Chow, E. P. F., Kong, F. Y. S., Hocking, J. S., Bissessor, M., Fairley, C. K., & Chen, M. (2016) · Emerging infectious diseases · doi:10.3201/eid2210.160986 · Cited by 31
Wood, G., Taylor, E., Ng, V., Murrell, A., Patil, A., van der Touw, T., Wolden, M., Andronicos, N., & Smart, N. A. (2023) · Sports Medicine · doi:10.1007/s40279-023-01817-0 · Cited by 31
“Included studies exceeded the minimum number of effect sizes to be pooled [98].”
BackgroundAntibiotic therapy for adults with neurosyphilis
Buitrago-Garcia, D., Martí-Carvajal, A. J., Jimenez, A., Conterno, L. O., & Pardo, R. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011399.pub2 · Cited by 30
Yan, Q., Zhu, C., Li, L., Li, Y., Chen, Y., & Hu, X. (2024) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2024.104895 · Cited by 30
Vidor, M. V., Panzenhagen, A. C., Martins, A. R., Cupertino, R. B., Bandeira, C. E., Picon, F. A., da Silva, B. S., Vitola, E. S., Rohde, L. A., Rovaris, D. L., et al. (2022) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-022-01397-6 · Cited by 29
Angeles-Hernandez, J. C., Vieyra Alberto, R., Kebreab, E., Appuhamy, J. A. D. R. N., Dougherty, H. C., Castelan-Ortega, O., & Gonzalez-Ronquillo, M. (2020) · Livestock Science · doi:10.1016/j.livsci.2020.104069 · Cited by 28
Millett, G., D'Amico, D., Amestoy, M. E., Gryspeerdt, C., & Fiocco, A. J. (2021) · Consciousness and Cognition · doi:10.1016/j.concog.2021.103195 · Cited by 25
Microbial contamination of packaged drinking water in Nigeria
Udoh, A., Lawal, B. K., Akpan, M., Labaran, K. S., Ndem, E., Ohabunwa, U., Tikare, O., Ibrahim, U. I., Amorha, K., & Kpokiri, E. (2021) · Tropical Medicine & International Health · doi:10.1111/tmi.13672 · Cited by 27
Yip, A. L. K., Karatzias, T., & Chien, W. T. (2022) · Annals of Medicine · doi:10.1080/07853890.2022.2108551 · Cited by 24
Shen, Y., Liu, D., Li, S., He, Y., Tan, F., Sun, X., Li, D., Xia, X., & Hao, Q. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.811746 · Cited by 23
Ahorsu, D. K., Adjaottor, E. S., & Lam, B. Y. H. (2021) · Brain Sciences · doi:10.3390/brainsci11070840 · Cited by 23
Michelet, D., Julien-Marsollier, F., Vacher, T., Bellon, M., Skhiri, A., Bruneau, B., Fournier, J., Diallo, T., Luce, V., Brasher, C., et al. (2019) · Sleep Medicine · doi:10.1016/j.sleep.2018.09.027 · Cited by 22
Maj, A., Michalak, N., Graczykowska, A., & Andersson, G. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100654 · Cited by 22
Maternal adiposity and perinatal and offspring outcomes: an umbrella review
Yang, Z., Feng, G., Gao, X., Yan, X., Li, Y., Wang, Y., Li, S., Jiang, Y., Zhao, S., Zhao, H., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01994-6 · Cited by 22
Reporting and methodological quality of meta-analyses in urological literature
Xia, L., Xu, J., & Guzzo, T. J. (2017) · PeerJ · doi:10.7717/peerj.3129 · Cited by 22
Davis, S. P., Bolin, L. P., Crane, P. B., Wei, H., & Crandell, J. (2020) · Complementary Therapies in Clinical Practice · doi:10.1016/j.ctcp.2020.101229 · Cited by 21
Ultrasound-induced immune responses in tumors: A systematic review and meta-analysis
Rix, A., Heinrichs, H., Porte, C., Leenaars, C., Bleich, A., & Kiessling, F. (2024) · Journal of Controlled Release · doi:10.1016/j.jconrel.2024.05.030 · Cited by 21
Wu, W., Liu, J., & Zhou, Z. (2020) · Emergency Medicine International · doi:10.1155/2020/1382475 · Cited by 20
“However, for each risk factor, no one included more than ten studies; these numbers were too small so that the test efficacy of funnel plots and egger’s test was insufficient [31].”
BackgroundRiley, T. M., Sapp, P. A., Kris-Etherton, P. M., & Petersen, K. S. (2024) · The American Journal of Clinical Nutrition · doi:10.1016/j.ajcnut.2024.06.019 · Cited by 20
Pham, C. H., Fang, M., Nager, J., Matsushima, K., Inaba, K., & Kuza, C. M. (2019) · Journal of Trauma and Acute Care Surgery · doi:10.1097/ta.0000000000002371 · Cited by 18
Hart, T. L., Damani, J. J., DiMattia, Z. S., Tate, K. E., Jafari, F., & Petersen, K. S. (2025) · Advances in Nutrition · doi:10.1016/j.advnut.2025.100502 · Cited by 16
“Publication bias was investigated by visual inspection of funnel plot asymmetry [24].”
Wang, Y., Chai, X., Wang, Y., Yin, X., Huang, X., Gong, Q., Zhang, J., Shao, R., & Li, G. (2025) · Journal of Medical Internet Research · doi:10.2196/63975 · Cited by 14
Riemma, G., García-Velasco, J. A., Abrão, M. S., Leone Roberti Maggiore, U., Vignali, M., Solima, E., Ceccaroni, M., Chiantera, V., Moawad, G. N., Laganà, A. S., et al. (2026) · Reproductive Biomedicine Online · doi:10.1016/j.rbmo.2025.105178 · Cited by 14
Yoshino, T., Taieb, J., Kuboki, Y., Pfeiffer, P., Kumar, A., & Hochster, H. S. (2023) · Therapeutic Advances in Medical Oncology · doi:10.1177/17588359221146137 · Cited by 12
Köhnen, M., Kriston, L., Härter, M., Dirmaier, J., & Liebherz, S. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028042 · Cited by 10
The Distributed Practice Effect on Classroom Learning: A Meta-Analytic Review of Applied Research
Mawson, R. D., & Kang, S. H. K. (2025) · Behavioral Sciences · doi:10.3390/bs15060771 · Cited by 14
“Instead, the asymmetry likely represents a ‘small study bias’, where methodological approaches inherent in small studies lead to stronger effects with greater variance (Peters et al., 2008; Sterne et al., 2011).”
Sun, Y., Tian, Z., Ma, Y., Zhu, Q., Zheng, R., Guan, Y., & Zhang, X. (2025) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/jpm.13138 · Cited by 12
Metadata
- Authors
- Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., Tetzlaff, J., Deeks, J. J., Peters, J., Macaskill, P., Schwarzer, G., Duval, S., Altman, D. G., Moher, D., & Higgins, J. P. T.
- Affiliations
- University of Bristol (GB)
- University of Leicester (GB)
- Stanford Medicine (US)
- Tufts University (US)
- London School of Hygiene & Tropical Medicine (GB)
- Ottawa Hospital Research Institute (CA)
- University of Birmingham (GB)
- University of Exeter (GB)
- The University of Sydney (AU)
- University of Freiburg (DE)
- University of Minnesota (US)
- University of Oxford (GB)
- Corresponding author
- Sterne, J. A. C.
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2011-07-22
- Volume
- 343
- Issue
- jul22 1
- Pages
- d4002-d4002
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- OpenAlex
- W2119605658
- Semantic Scholar
- 660168dbe48731c1dae7dfd14a0525404d8e59e5
- Cited by
- 7,709
- References
- 45
- Citations by year
- 2026: 967 · 2025: 792 · 2024: 824 · 2023: 766 · 2022: 786 · 2021: 687 · 2020: 610 · 2019: 492 · 2018: 447 · 2017: 377 · 2016: 304 · 2015: 241 · 2014: 197 · 2013: 122 · 2012: 74
- FWCI
- 136.80
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- Leicester Research Archive (University of Leicester) (submittedVersion)
- HighWire Press Open Archive (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Explore Bristol Research (submittedVersion)
- Funders
- Cancer Research UK
- Deutsche Forschungsgemeinschaft (FOR 534)
- Canadian Institutes of Health Research
- Medical Research Council (U1052.00.011)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- funnel plot asymmetry, publication bias, meta-analysis bias, randomized controlled trials, heterogeneity
- MeSH terms
- Statistics as Topic, Meta-Analysis as Topic, Selection Bias, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
