The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey
Open accessAbstract
BACKGROUND: Statistical tests for funnel-plot asymmetry are common in meta-analyses. Inappropriate application can generate misleading inferences about publication bias. We aimed to measure, in a survey of meta-analyses, how frequently the application of these tests would be not meaningful or inappropriate. METHODS: We evaluated all meta-analyses of binary outcomes with é 3 studies in the Cochrane Database of Systematic Reviews (2003, issue 2). A separate, restricted analysis was confined to the largest meta-analysis in each of the review articles. In each meta-analysis, we assessed whether criteria to apply asymmetry tests were met: no significant heterogeneity, I2 < 50%, é 10 studies (with statistically significant results in at least 1) and ratio of the maximal to minimal variance across studies > 4. We performed a correlation and 2 regression asymmetry tests and evaluated their concordance. Finally, we sampled 60 meta-analyses from print journals in 2005 that cited use of the standard regression test. RESULTS: A total of 366 of 6873 (5%) and 98 of 846 meta-analyses (12%) in the wider and restricted Cochrane data set, respectively, would have qualified for use of asymmetry tests. Asymmetry test results were significant in 7%-18% of the meta-analyses. Concordance between the 3 tests was modest (estimated k 0.33-0.66). Of the 60 journal meta-analyses, 7 (12%) would qualify for asymmetry tests; all 11 claims for identification of publication bias were made in the face of large and significant heterogeneity. INTERPRETATION: Statistical conditions for employing asymmetry tests for publication bias are absent from most meta-analyses; yet, in medical journals these tests are performed often and interpreted erroneously.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, publication bias tests might be problematic for IN-OT studies due to small sample sizes resulting in insufficient statistical power and sample size variability for the tests to yield reliable and significant results (26).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [26]
Experimental — accuracy not yet measuredIn the citing paper
“The small number of studies and presence of significant heterogeneity precluded formal testing for publication bias[35].”
From Daily Sitting Time and All-Cause Mortality: A Meta-Analysis (2013) · cites this paper as [35]
In the citing paper
“Of 60 meta-analyses in 24 articles published in 2005 in which formal assessments were reported, most were based on fewer than ten studies; most displayed statistically significant heterogeneity; and many reviewers misinterpreted the results of the tests employed [123].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [123]
In the citing paper
“Although evidence that smaller studies had larger estimated effects than large ones may suggest the possibility that the available evidence is biased, misinterpretation of such data is common [123].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [123]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,232
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Self-citationFunnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109
Cited by
Sorted by citations
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationSterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
“After reviewing the results of these studies, and based on theoretical considerations, we formulated recommendations on testing for funnel plot asymmetry (box 2). The appendix describes the proposed tests, explains the reasons that some were not recommended, and discusses funnel plots for intervention effects measured…”
MethodSelf-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Italian Journal of Public Health · doi:10.2427/5768 · Cited by 3,374
Self-citationSystematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,270
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542
Undue reliance on I 2 in assessing heterogeneity may mislead
Rücker, G., Schwarzer, G., Carpenter, J. R., & Schumacher, M. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-79 · Cited by 1,287
Griesdale, D. E. G., de Souza, R. J., van Dam, R. M., Heyland, D. K., Cook, D. J., Malhotra, A., Dhaliwal, R., Henderson, W. R., Chittock, D. R., Finfer, S., et al. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.090206 · Cited by 1,096
“bias in our analysis, the tests we used may be unreliable in the presence of significant heterogeneity.(41) However, since neither peer reviewers nor we have been able to assess the methodologic quality of the unpublished trials, we decided to include only trials published either in full or as abstracts in indexed jou…”
MethodThe effect of testing versus restudy on retention: A meta-analytic review of the testing effect.
Rowland, C. A. (2014) · Psychological Bulletin · doi:10.1037/a0037559 · Cited by 1,066
Dissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039
Chapter 10: Addressing reporting biases
Egger, M., & Moher, D. (2011) · Cited by 977
Jonas, D. E., Amick, H. R., Feltner, C., Bobashev, G., Thomas, K., Wines, R., Kim, M. M., Shanahan, E., Gass, C. E., Rowe, C. J., et al. (2014) · JAMA · doi:10.1001/jama.2014.3628 · Cited by 923
Daily Sitting Time and All-Cause Mortality: A Meta-Analysis
Chau, J. Y., Grunseit, A. C., Chey, T., Stamatakis, E., Brown, W. J., Matthews, C. E., Bauman, A. E., & van der Ploeg, H. P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0080000 · Cited by 876
“Evidence of publication bias The small number of studies and presence of significant heterogeneity precluded formal testing for publication bias[35].”
BackgroundShi, L., & Lin, L. (2019) · Medicine · doi:10.1097/md.0000000000015987 · Cited by 818
“the funnel plot’s asymmetry is truly caused by publication bias or confounded by heterogeneity.([49]) More sophisticated statistical methods (e.”
MethodZachariae, R., Lyby, M. S., Ritterband, L. M., & O'Toole, M. S. (2016) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2015.10.004 · Cited by 704
Jakobsen, J. C., Wetterslev, J., Winkel, P., Lange, T., & Gluud, C. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-120 · Cited by 704
“If there are no trials showing statistically significant effects, then publication bias may not be a plausible explanation for funnel plot asymmetry [13,71].”
BackgroundInterpretation of tests of heterogeneity and bias in meta-analysis
Ioannidis, J. P. A. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.00986.x · Cited by 696
Self-citationBeaudart, C., Buckinx, F., Rabenda, V., Gillain, S., Cavalier, E., Slomian, J., Petermans, J., Reginster, J.-Y., & Bruyère, O. (2014) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2014-1742 · Cited by 676
“Moreover, because of the limited number of studies included in the meta-analyses on muscle mass and muscle power and because of the high heterogeneity observed in the metaanalysis on muscle strength, we were unable to measure the potential publication bias by the Begg’s adjusted rank correlation and the Egger’s regres…”
MethodMeta-Analyst: software for meta-analysis of binary, continuous and diagnostic data
Wallace, B., Schmid, C. H., Lau, J., & Trikalinos, T. A. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-80 · Cited by 663
Self-citationFerguson, C. J., & Heene, M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459059 · Cited by 603
Probiotics for the prevention of pediatric antibiotic-associated diarrhea
Goldenberg, J. Z., Lytvyn, L., Steurich, J., Parkin, P. C., Mahant, S., & Johnston, B. C. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004827.pub4 · Cited by 635
Hooper, L., Kay, C., Abdelhamid, A., Kroon, P. A., Cohn, J. S., Rimm, E. B., & Cassidy, A. (2012) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.111.023457 · Cited by 610
Risk of Clostridium difficile Infection With Acid Suppressing Drugs and Antibiotics: Meta-Analysis
Kwok, C. S., Arthur, A. K., Anibueze, C. I., Singh, S., Cavallazzi, R., & Loke, Y. K. (2012) · The American Journal of Gastroenterology · doi:10.1038/ajg.2012.108 · Cited by 579
Long-term Glycemic Variability and Risk of Adverse Outcomes: A Systematic Review and Meta-analysis
Gorst, C., Kwok, C. S., Aslam, S., Buchan, I., Kontopantelis, E., Myint, P. K., Heatlie, G., Loke, Y., Rutter, M. K., & Mamas, M. A. (2015) · Diabetes Care · doi:10.2337/dc15-1188 · Cited by 530
“10 studies and no evidence of statistical heterogeneity in a particular metaanalysis (34).”
BackgroundA meta-analysis of predictors of offender treatment attrition and its relationship to recidivism.
Olver, M. E., Stockdale, K. C., & Wormith, J. S. (2011) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0022200 · Cited by 524
Detecting and avoiding likely false‐positive findings – a practical guide
Forstmeier, W., Wagenmakers, E., & Parker, T. H. (2017) · Biological Reviews · doi:10.1111/brv.12315 · Cited by 436
Empirical Comparison of Publication Bias Tests in Meta-Analysis
Lin, L., Chu, H., Murad, M. H., Hong, C., Qu, Z., Cole, S. R., & Chen, Y. (2018) · Journal of General Internal Medicine · doi:10.1007/s11606-018-4425-7 · Cited by 464
Serum and plasma BDNF levels in major depression: A replication study and meta-analyses
Bocchio-Chiavetto, L., Bagnardi, V., Zanardini, R., Molteni, R., Gabriela Nielsen, M., Placentino, A., Giovannini, C., Rillosi, L., Ventriglia, M., Riva, M. A., et al. (2010) · The World Journal of Biological Psychiatry · doi:10.3109/15622971003611319 · Cited by 435
“In the presence of heterogeneity, no attempt to evaluate publication bias was done (Ioannidis and Trikalinos 2007).”
BackgroundMeta-analysis of filaggrin polymorphisms in eczema and asthma: Robust risk factors in atopic disease
Rodríguez, E., Baurecht, H., Herberich, E., Wagenpfeil, S., Brown, S., Cordell, H. J., Irvine, A. D., & Weidinger, S. (2009) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2009.03.036 · Cited by 434
Tauber, N. M., O’Toole, M. S., Dinkel, A., Galica, J., Humphris, G., Lebel, S., Maheu, C., Ozakinci, G., Prins, J., Sharpe, L., et al. (2019) · Journal of Clinical Oncology · doi:10.1200/jco.19.00572 · Cited by 402
“Publication Bias Positive and negative findings are not equally likely to be published, and publication bias is a widespread problem when reviewing available evidence.(32) We evaluated publication bias using funnel plots and Egger’s test.”
MethodPicetti, R., Deeney, M., Pastorino, S., Miller, M. R., Shah, A. S. V., Leon, D. A., Dangour, A. D., & Green, R. (2022) · Environmental Research · doi:10.1016/j.envres.2022.112988 · Cited by 430
“…per cancer type and study design meant that we did not fulfil at least one of the four conditions for reliably using asymmetry tests for publication bias, i.e. ≥ 10 studies (Ioannidis and Trikalinos, 2007), or a minimum of 10 studies per moderator to run a meta-regression (Deeks et al., 2019).”
Wozniak, D. R., Lasserson, T. J., & Smith, I. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007736.pub2 · Cited by 388
Frederiksen, Y., Farver-Vestergaard, I., Skovgård, N. G., Ingerslev, H. J., & Zachariae, R. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-006592 · Cited by 380
Moreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-2 · Cited by 373
Oxidative stress-related biomarkers in autism: Systematic review and meta-analyses
Frustaci, A., Neri, M., Cesario, A., Adams, J. B., Domenici, E., Bernardina, B. D., & Bonassi, S. (2012) · Free Radical Biology and Medicine · doi:10.1016/j.freeradbiomed.2012.03.011 · Cited by 383
“Visual analysis of funnel plots and Egger’s test revealed no publication bias, but it must be taken into account the low reliability of these methods when the number of included studies is small [98].”
MethodGriesdale, D. E. G., Liu, D., McKinney, J., & Choi, P. T. (2011) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-011-9620-5 · Cited by 361
“8) Sun 2005(15) 75/24/1/0 59/26/15/0 94/100 (94%) 97/100 (97%) 46 (43-49) 30 (28-33) Teoh 2009(27) 78/21/1/0 58/37/5/0 91/100 (91%) 98/100 (98%) 31.”
Tierney, J. F., Vale, C. L., Riley, R. D., Smith, C. T., Stewart, L. A., Clarke, M. J., & Rovers, M. M. (2015) · PLoS Medicine · doi:10.1371/journal.pmed.1001855 · Cited by 362
“Funnel plots may also help indicate the potential impact of both known and unknown missing trials on the results [25] but would not be appropriate in a meta-analysis including less than ten trials [25] or few trials with statistically significant results or one in which heterogeneity has been observed [26].”
Sensitivity Analysis for Publication Bias in Meta-Analyses
Mathur, M. B., & VanderWeele, T. J. (2020) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/rssc.12440 · Cited by 354
“In practice, most meta-analyses are far too small to apply selection models; for example, the percentage of meta-analyses comprising fewer than 10 studies probably exceeds 50% in the Cochrane database (Ioannidis and Trikalinos, 2007) and in medical journals (Sterne et al., 2000).”
BackgroundPrevalence and factors associated with polypharmacy: a systematic review and meta-analysis
Delara, M., Murray, L., Jafari, B., Bahji, A., Goodarzi, Z., Kirkham, J. G., Chowdhury, M., & Seitz, D. P. (2022) · BMC Geriatrics · doi:10.1186/s12877-022-03279-x · Cited by 348
“Publication bias was assessed for the prevalence of polypharmacy using funnel plots and Egger’s test [9].”
MethodStatistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies
Walum, H., Waldman, I. D., & Young, L. J. (2016) · Biological Psychiatry · doi:10.1016/j.biopsych.2015.06.016 · Cited by 347
Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215052 · Cited by 365
“Homogeneous subsets were created, because publication bias methods have poor statistical properties if the true effect size is heterogeneous [5, 32, 50, 52].”
BackgroundMethodGannon, T. A., Olver, M. E., Mallion, J. S., & James, M. (2019) · Clinical Psychology Review · doi:10.1016/j.cpr.2019.101752 · Cited by 335
Bariatric surgery and cardiovascular disease: a systematic review and meta-analysis
van Veldhuisen, S. L., Gorter, T. M., van Woerden, G., de Boer, R. A., Rienstra, M., Hazebroek, E. J., & van Veldhuisen, D. J. (2022) · European Heart Journal · doi:10.1093/eurheartj/ehac071 · Cited by 322
“Publication bias was tested with funnel plot asymmetry and Egger’s regression test if a minimum of 10 studies was included in the analysis.(38,39) Inter-rater agreement for the quality assessment was tested using Cohen’s kappa coefficient.”
Köhler, C. A., Evangelou, E., Stubbs, B., Solmi, M., Veronese, N., Belbasis, L., Bortolato, B., Melo, M. C. A., Coelho, C. A., Fernandes, B. S., et al. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.05.020 · Cited by 302
Self-citationExcess Significance Bias in the Literature on Brain Volume Abnormalities
Ioannidis, J. P. A. (2011) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.28 · Cited by 299
Self-citationKotsifaki, R., Korakakis, V., King, E., Barbosa, O., Maree, D., Pantouveris, M., Bjerregaard, A., Luomajoki, J., Wilhelmsen, J., & Whiteley, R. (2023) · British Journal of Sports Medicine · doi:10.1136/bjsports-2022-106158 · Cited by 303
Evaluation of excess significance bias in animal studies of neurological diseases
Tsilidis, K. K., Panagiotou, O. A., Sena, E. S., Aretouli, E., Evangelou, E., Howells, D. W., Salman, R. A.-S., Macleod, M. R., & Ioannidis, J. P. A. (2013) · PLoS Biology · doi:10.1371/journal.pbio.1001609 · Cited by 295
“However, these methods may not be very sensitive or specific in the detection of such biases, especially when few studies are included in a meta-analysis [11–13].”
MethodSelf-citationTreatment-effect estimates adjusted for small-study effects via a limit meta-analysis
Rücker, G., Schwarzer, G., Carpenter, J. R., Binder, H., & Schumacher, M. C. (2010) · Biostatistics · doi:10.1093/biostatistics/kxq046 · Cited by 295
“However, when applied following a prespecified analysis protocol, with their limitations duly acknowledged, we argue that such concerns (Ioannidis and Trikalinos, 2007) are minimized and that testing is a useful aide to researchers in judging funnel plots.”
Atrial fibrillation and incidence of dementia
Kwok, C. S., Loke, Y. K., Hale, R., Potter, J. F., & Myint, P. K. (2011) · Neurology · doi:10.1212/wnl.0b013e31820f2e38 · Cited by 286
Effectiveness of disease-management programs for improving diabetes care: a meta-analysis
Pimouguet, C., Goff, M. L., Thiébaut, R., Jf, D., & Helmer, C. (2010) · Canadian Medical Association Journal · doi:10.1503/cmaj.091786 · Cited by 276
Loneliness in Psychosis: A Meta-analytical Review
da Rocha, B. M., Rhodes, S., Vasilopoulou, E., & Hutton, P. (2017) · Schizophrenia Bulletin · doi:10.1093/schbul/sbx036 · Cited by 310
PD-L1 and gastric cancer prognosis: A systematic review and meta-analysis
Gu, L., Chen, M., Guo, D., Zhu, H., Zhang, W., Pan, J., Zhong, X., Li, X., Qian, H., & Wang, X. (2017) · PLoS ONE · doi:10.1371/journal.pone.0182692 · Cited by 286
Kuester, A., Niemeyer, H., & Knaevelsrud, C. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.11.004 · Cited by 261
Publication bias: what is it? How do we measure it? How do we avoid it?
Song, F., Hooper, L., & Loke, Y. K. (2013) · Open Access Journal of Clinical Trials · doi:10.2147/oajct.s34419 · Cited by 298
MacArthur, G. J., Harrison, S., Caldwell, D. M., Hickman, M., & Campbell, R. (2016) · Addiction · doi:10.1111/add.13224 · Cited by 265
Marital status and risk of cardiovascular diseases: a systematic review and meta-analysis
Wong, C. W., Kwok, C. S., Narain, A., Gulati, M., Mihailidou, A. S., Wu, P., Alasnag, M. A., Myint, P. K., & Mamas, M. A. (2018) · Heart · doi:10.1136/heartjnl-2018-313005 · Cited by 271
“plots if there is no evidence of significant heterogeneity.[23] Further subgroup analyses were”
BackgroundContract farming for improving smallholder incomes: What can we learn from effectiveness studies?
Ton, G., Vellema, W., Desiere, S., Weituschat, S., & D'Haese, M. (2018) · World Development · doi:10.1016/j.worlddev.2017.11.015 · Cited by 275
Peery, A. F., Kelly, C. R., Kao, D. H., Vaughn, B. P., Lebwohl, B., Singh, S. S., Imdad, A., & Altayar, O. (2024) · Gastroenterology · doi:10.1053/j.gastro.2024.01.008 · Cited by 299
Meta-analysis using effect size distributions of only statistically significant studies.
van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015) · Psychological Methods · doi:10.1037/met0000025 · Cited by 265
Kwok, C. S., Pradhan, A. K., Khan, M. A., Anderson, S. G., Keavney, B. D., Myint, P. K., Mamas, M. A., & Loke, Y. K. (2014) · International Journal of Cardiology · doi:10.1016/j.ijcard.2014.02.026 · Cited by 270
Patel, S. H., Wang, Z., Wong, W. W., Murad, M. H., Buckey, C. R., Mohammed, K. S., Alahdab, F., Altayar, O., Nabhan, M., Schild, S. E., et al. (2014) · The Lancet Oncology · doi:10.1016/s1470-2045(14)70268-2 · Cited by 263
Invited review: Use of meta-analysis in animal health and reproduction: Methods and applications
Lean, I. J., Rabiee, A. R., Duffield, T. F., & Dohoo, I. R. (2009) · Journal of Dairy Science · doi:10.3168/jds.2009-2140 · Cited by 265
“If there are no statistically significant studies, then publication bias may not be a plausible explanation for funnel plot asymmetry (Ioannidis and Trikalinos, 2007).”
BackgroundProbiotics for the prevention of pediatric antibiotic-associated diarrhea
Johnston, B. C., Goldenberg, J. Z., Vandvik, P. O., Sun, X., & Guyatt, G. H. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004827.pub3 · Cited by 264
Computerized advice on drug dosage to improve prescribing practice
Durieux, P., Trinquart, L., Colombet, I., Niès, J., Walton, R. T., Rajeswaran, A., Rège-Walther, M., Harvey, E., & Burnand, B. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002894.pub2 · Cited by 251
Kopterides, P., Siempos, I. Ι., Tsangaris, I., Tsantes, A. Ε., & Armaganidis, A. (2010) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181f17bf9 · Cited by 243
“As a result of the small number of the included trials, detection of publication bias (29) and metaregression analysis (30) to explore sources of heterogeneity among studies were not performed.”
BackgroundCarter, E. C., & McCullough, M. E. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00823 · Cited by 381
Revisiting the serotonin–aggression relation in humans: A meta-analysis.
Duke, A. A., Bègue, L., Bell, R., & Eisenlohr-Moul, T. A. (2013) · Psychological Bulletin · doi:10.1037/a0031544 · Cited by 253
“All four criteria were met for statistically testing funnel plot asymmetry (see Ioannidis & Trikalinos, 2007).”
Prophylactic pancreatic stent placement and post-ERCP pancreatitis: an updated meta-analysis
Mazaki, T., Mado, K., Masuda, H., & Shiono, M. (2013) · Journal of Gastroenterology · doi:10.1007/s00535-013-0806-1 · Cited by 236
“10 was considered to be significant [38].”
Narain, A., Kwok, C. S., & Mamas, M. A. (2016) · International Journal of Clinical Practice · doi:10.1111/ijcp.12841 · Cited by 223
Nikolopoulos, G. K., Paraskevis, D. N., Hatzitheodorou, E., Moschidis, Z., Sypsa, V., Zavitsanos, X., Kalapothaki, V., & Hatzakis, A. E. (2009) · Clinical Infectious Diseases · doi:10.1086/599110 · Cited by 230
“The appropriateness of the asymmetry tests was checked using criteria reported elsewhere [23].”
MethodMeta‐research: The art of getting it wrong
Ioannidis, J. P. A. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.19 · Cited by 237
“2010, 1 169--184 1 7 5 A number of tests have been developed by meta-analysts to detect and exclude publication bias, but I am afraid that they are used beyond their real capacity (Ioannidis, 2008a; Ioannidis and Trikalinos, 2007b; Lau et al., 2006).”
BackgroundInfluentialSelf-citationCatalogue of bias: publication bias
DeVito, N., & Goldacre, B. M. (2018) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2018-111107 · Cited by 247
Johannsen, M., Damholdt, M. F., Zachariae, R., Lundorff, M., Farver-Vestergaard, I., & O'Connor, M. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.04.065 · Cited by 214
How evidence‐based medicine is failing due to biased trials and selective publication
Every‐Palmer, S., & Howick, J. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12147 · Cited by 240
“Numerous methods are available, but none can identify or rule out selection bias entirely [54,58–61].”
MethodEfficacy and Safety of Finasteride Therapy for Androgenetic Alopecia
Mella, J. M., Perret, M. C., Manzotti, M., Catalano, H. N., & Guyatt, G. H. (2010) · Archives of Dermatology · doi:10.1001/archdermatol.2010.256 · Cited by 221
Methods for meta-analysis in genetic association studies: a review of their potential and pitfalls
Kavvoura, F. K., & Ioannidis, J. P. A. (2007) · Human Genetics · doi:10.1007/s00439-007-0445-9 · Cited by 209
“Unfortunately, in most cases these tests are used either inappropriately or meaninglessly (Ioannidis and Trikalinos 2007a).”
BackgroundSelf-citationRisk factors for severe perineal lacerations during childbirth
Pergialiotis, V., Vlachos, D., Πρωτόπαπας, Α., Pappa, K. I., & Vlachos, G. D. (2014) · International Journal of Gynecology & Obstetrics · doi:10.1016/j.ijgo.2013.09.034 · Cited by 211
“For indices that were evaluated in more than 10 studies, publication bias was assessed with the Egger test [8], using Comprehensive MetaAnalysis version 2 (Biostat, Englewood, NJ, USA).”
MethodWu, B., Gu, D.-Z., Yu, J., Yang, J., & Shen, W. (2020) · Infection Genetics and Evolution · doi:10.1016/j.meegid.2020.104485 · Cited by 217
Siempos, I. I., Ntaidou, T. K., & Falagas, M. E. (2010) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181c8fe4b · Cited by 204
“Given that the number of included trials was 10, detection of publication bias was considered meaningless (12).”
BackgroundSchoenleber, S. J., Kurtz, D. M., Talwalkar, J. A., Roberts, L. R., & Gores, G. J. (2009) · British Journal of Cancer · doi:10.1038/sj.bjc.6605017 · Cited by 207
“We also focused on using one statistical approach, as performing multiple tests can generate discordant results that confuse data interpretation (Ioannidis and Trikalinos, 2007).”
ResultKwok, C. S., Umar, S., Myint, P. K., Mamas, M. A., & Loke, Y. K. (2014) · International Journal of Cardiology · doi:10.1016/j.ijcard.2014.07.080 · Cited by 205
“If there were N10 studies available in themeta-analysis, with no evidence of substantial statistical heterogeneity, we aimed to generate funnel plots to assess the possibility of publication bias [15].”
MethodWu, P., Gulati, M., Kwok, C. S., Wong, C. W., Narain, A., O'Brien, S., Chew‐Graham, C. A., Verma, G., Kadam, U. T., & Mamas, M. A. (2018) · Journal of the American Heart Association · doi:10.1161/jaha.117.007809 · Cited by 204
“In the case of an analysis where there are more than 10 studies and little evidence of heterogeneity, we planned to perform funnel plots to assess for publication bias.(40) Sensitivity analysis was performed to consider the follow-up duration of the studies (<10, 10–30, and >30 years), gestation (<32 weeks versus 32–3…”
MethodVitamin C for preventing and treating pneumonia
Hemilä, H., & Louhiala, P. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005532.pub3 · Cited by 205
Meta-analysis: Combination Endoscopic and Drug Therapy to Prevent Variceal Rebleeding in Cirrhosis
González, R., Zamora, J., Gómez‐Camarero, J., Molinero, L.-M., Bañares, R., & Albillos, A. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-2-200807150-00007 · Cited by 203
Katritsis, D. G., Siontis, G. C. M., Kastrati, A., Hof, A. W. J. V. ’., Neumann, F.-J., Siontis, K. C. M., & Ioannidis, J. P. A. (2010) · European Heart Journal · doi:10.1093/eurheartj/ehq276 · Cited by 199
“We did not perform funnel plot asymmetry tests since they are inappropriate when only four trials are available.(16)”
MethodSelf-citationReview and meta-analysis of ab-interno trabeculectomy outcomes
Kaplowitz, K. B., Bussel, I. I., Honkanen, R., Schuman, J. S., & Loewen, N. A. (2016) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2015-307131 · Cited by 197
Tse, F., & Yuan, Y. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009779.pub2 · Cited by 195
A meta-analysis of factors related to recycling
Geiger, J. L., Steg, L., van der Werff, E., & Ünal, A. B. (2019) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2019.05.004 · Cited by 219
Optimum duration of regimens for Helicobacter pylori eradication
Yuan, Y., Ford, A. C., Khan, K. J., Gisbert, J. P., Forman, D., Leontiadis, G. I., Tse, F., Calvet, X., Fallone, C. A., Fischbach, L. A., et al. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008337.pub2 · Cited by 195
Generalized Joint Hypermobility and Risk of Lower Limb Joint Injury During Sport
Pacey, V., Nicholson, L. L., Adams, R. D., Munn, J., & Munns, C. F. (2010) · The American Journal of Sports Medicine · doi:10.1177/0363546510364838 · Cited by 193
The Efficacy and Safety of Heparin in Patients With Sepsis
Zarychanski, R., Abou-Setta, A. M., Kanji, S., Turgeon, A. F., Kumar, A., Houston, D. S., Rimmer, E. K., Houston, B. L., McIntyre, L., Fox‐Robichaud, A., et al. (2014) · Critical Care Medicine · doi:10.1097/ccm.0000000000000763 · Cited by 190
Regnaux, J.-P., Lefevre-Colau, M.-M., Trinquart, L., Nguyen, C., Boutron, I., Brosseau, L., & Ravaud, P. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010203.pub2 · Cited by 184
Aid and Growth: What Meta-Analysis Reveals
Mekasha, T. J., & Tarp, F. (2013) · The Journal of Development Studies · doi:10.1080/00220388.2012.709621 · Cited by 195
Cognitive behavioural therapy for psychosis prevention: a systematic review and meta-analysis
Hutton, P., & Taylor, P. J. (2014) · Psychological Medicine · doi:10.1017/s0033291713000354 · Cited by 184
Polyzos, N. P., Polyzos, I. P., Zavos, A., Valachis, A., Mauri, D. N., Papanikolaou, E. G. E., Tzioras, S., Weber, D., & Messinis, I. (2010) · BMJ · doi:10.1136/bmj.c7017 · Cited by 186
“Thus, early trials that did not favour treatment of periodontal with scaling and root planing during pregnancy may have been left unpublished, resulting in the accumulation of small trials with significant results in favour of treatment. According to the funnel plot constructed, publication bias is highly likely to ex…”
ResultMoreno, S. G., Sutton, A. J., Turner, E. H., Abrams, K. R., Cooper, N. J., Palmer, T. M., & Ades, A. E. (2009) · BMJ · doi:10.1136/bmj.b2981 · Cited by 184
Pharmacologic Interventions for Painful Diabetic Neuropathy
Griebeler, M. L., Morey‐Vargas, O. L., Brito, J. P., Τσάπας, Α., Wang, Z., Leon, B. G. C., Phung, O. J., Montori, V. M., & Murad, M. H. (2014) · Annals of Internal Medicine · doi:10.7326/m14-0511 · Cited by 184
“We standardized the pain scales used in studies by estimating SMDs using the Cohen d method and 95% credible intervals (CrIs) (10).”
BackgroundMethod
Metadata
- Authors
- Ioannidis, J. P. A., & Trikalinos, T. A.
- Affiliations
- Tufts University (US)
- Venue
- Canadian Medical Association Journal
- Publisher
- CMA Joule Inc.
- Published
- 2007-04-10
- Volume
- 176
- Issue
- 8
- Pages
- 1091-1096
- ISSN
- 0820-3946 (ISSN-L)
- 1488-2329
- PMC
- PMC1839799
- OpenAlex
- W2164093802
- Cited by
- 1,052
- References
- 40
- Citations by year
- 2026: 49 · 2025: 48 · 2024: 64 · 2023: 49 · 2022: 81 · 2021: 73 · 2020: 83 · 2019: 81 · 2018: 73 · 2017: 55 · 2016: 62 · 2015: 64 · 2014: 68 · 2013: 38 · 2012: 39
- FWCI
- 31.66
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Canadian Medical Association Journal (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, funnel plot asymmetry, meta-analysis, heterogeneity, Cochrane Database of Systematic Reviews, binary outcomes, test concordance
- MeSH terms
- Data Interpretation, Statistical, Humans, Periodicals as Topic, Meta-Analysis as Topic, Databases, Bibliographic, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
