The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey
Open accessAbstract
BACKGROUND: Statistical tests for funnel-plot asymmetry are common in meta-analyses. Inappropriate application can generate misleading inferences about publication bias. We aimed to measure, in a survey of meta-analyses, how frequently the application of these tests would be not meaningful or inappropriate. METHODS: We evaluated all meta-analyses of binary outcomes with é 3 studies in the Cochrane Database of Systematic Reviews (2003, issue 2). A separate, restricted analysis was confined to the largest meta-analysis in each of the review articles. In each meta-analysis, we assessed whether criteria to apply asymmetry tests were met: no significant heterogeneity, I2 < 50%, é 10 studies (with statistically significant results in at least 1) and ratio of the maximal to minimal variance across studies > 4. We performed a correlation and 2 regression asymmetry tests and evaluated their concordance. Finally, we sampled 60 meta-analyses from print journals in 2005 that cited use of the standard regression test. RESULTS: A total of 366 of 6873 (5%) and 98 of 846 meta-analyses (12%) in the wider and restricted Cochrane data set, respectively, would have qualified for use of asymmetry tests. Asymmetry test results were significant in 7%-18% of the meta-analyses. Concordance between the 3 tests was modest (estimated k 0.33-0.66). Of the 60 journal meta-analyses, 7 (12%) would qualify for asymmetry tests; all 11 claims for identification of publication bias were made in the face of large and significant heterogeneity. INTERPRETATION: Statistical conditions for employing asymmetry tests for publication bias are absent from most meta-analyses; yet, in medical journals these tests are performed often and interpreted erroneously.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, publication bias tests might be problematic for IN-OT studies due to small sample sizes resulting in insufficient statistical power and sample size variability for the tests to yield reliable and significant results (26).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [26]
Experimental — accuracy not yet measuredIn the citing paper
“The small number of studies and presence of significant heterogeneity precluded formal testing for publication bias[35].”
From Daily Sitting Time and All-Cause Mortality: A Meta-Analysis (2013) · cites this paper as [35]
In the citing paper
“Of 60 meta-analyses in 24 articles published in 2005 in which formal assessments were reported, most were based on fewer than ten studies; most displayed statistically significant heterogeneity; and many reviewers misinterpreted the results of the tests employed [123].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [123]
In the citing paper
“Although evidence that smaller studies had larger estimated effects than large ones may suggest the possibility that the available evidence is biased, misinterpretation of such data is common [123].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [123]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,232
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Self-citationFunnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109
Cited by
Sorted by citations
Items 201–300 of 1,000
Systematic Meta-Analyses and Field Synopsis of Genetic Association Studies in Colorectal Cancer
Τheodoratou, E., Montazeri, Z., Hawken, S., Allum, G. C., Gong, J., Tait, V., Kirac, I., Tazari, M., Farrington, S. M., Demarsh, A., et al. (2012) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djs369 · Cited by 97
Psychological stress and psoriasis: a systematic review and meta-analysis
Snast, I., Reiter, O., Atzmony, L., Leshem, Y. A., Hodak, E., Mimouni, D., & Pavlovsky, L. (2017) · British Journal of Dermatology · doi:10.1111/bjd.16116 · Cited by 100
Ibrahim, T., Qureshi, A., Sutton, A. J., & Dias, J. J. (2011) · The Journal Of Hand Surgery · doi:10.1016/j.jhsa.2011.08.033 · Cited by 95
Browning, B. L., Hüebner, C., Petermann, I., Gearry, R. B., Barclay, M. L., Shelling, A. N., & Ferguson, L. R. (2007) · The American Journal of Gastroenterology · doi:10.1111/j.1572-0241.2007.01463.x · Cited by 95
Interventions for chronic palmoplantar pustulosis
Obeid, G., Do, G., Kirby, L. C., Hughes, C., Sbidian, É., & Cleach, L. L. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011628.pub2 · Cited by 93
Wood, L., Byrne, R., Varese, F., & Morrison, A. P. (2016) · Schizophrenia Research · doi:10.1016/j.schres.2016.05.001 · Cited by 93
Tsilidis, K. K., Papatheodorou, S., Εvangelou, Ε., & Ioannidis, J. P. A. (2012) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djs437 · Cited by 92
Self-citationSoft drink intake and the risk of metabolic syndrome: A systematic review and meta-analysis
Narain, A., Kwok, C. S., & Mamas, M. A. (2017) · International Journal of Clinical Practice · doi:10.1111/ijcp.12927 · Cited by 95
Barco, S., Ende-Verhaar, Y. M., Becattini, C., Jiménez, D., Lankeit, M., Huisman, M. V., Konstantinides, S., & Klok, F. A. (2018) · European Heart Journal · doi:10.1093/eurheartj/ehy631 · Cited by 100
Human papillomavirus type 16 and 18 in primary lung cancers--a meta-analysis
Srinivasan, M., Taioli, E., & Ragin, C. C. R. (2009) · Carcinogenesis · doi:10.1093/carcin/bgp177 · Cited by 92
Cunnington, C., Kwok, C. S., Satchithananda, D. K., Patwala, A., Khan, M. A., Zaidi, A. M., Ahmed, F. Z., & Mamas, M. A. (2015) · Heart · doi:10.1136/heartjnl-2014-306811 · Cited by 96
Three meta-analyses of children's emotion knowledge and their school success
Voltmer, K., & von Salisch, M. (2017) · Learning and Individual Differences · doi:10.1016/j.lindif.2017.08.006 · Cited by 108
Bevacizumab in metastatic breast cancer: a meta-analysis of randomized controlled trials
Valachis, A., Polyzos, N. P., Patsopoulos, N. A., Georgoulias, V., Mavroudis, D., & Mauri, D. N. (2010) · Breast Cancer Research and Treatment · doi:10.1007/s10549-009-0727-0 · Cited by 91
Generic versus brand-name drugs used in cardiovascular diseases
Manzoli, L., Flacco, M. E., Boccia, S., D’Andrea, E., Panić, N., Marzuillo, C., Siliquini, R., Ricciardi, W., Villari, P., & Ioannidis, J. P. A. (2015) · European Journal of Epidemiology · doi:10.1007/s10654-015-0104-8 · Cited by 93
“In case of limited data I(2) metric 95 % confidence intervals are typically large [58] and thus inferences on the magnitude of the statistical heterogeneity should be cautious.”
BackgroundSelf-citationPhysical activity and incidence of atrial fibrillation: A systematic review and meta-analysis
Kwok, C. S., Anderson, S. G., Myint, P. K., Mamas, M. A., & Loke, Y. K. (2014) · International Journal of Cardiology · doi:10.1016/j.ijcard.2014.09.104 · Cited by 89
“studies in the meta-analysis, and if statistical heterogeneity was <50%.[24] Data analysis We used RevMan 5.”
Karageorgiou, V., Papaioannou, T., Bellos, I., Alexandraki, K., Tentolouris, N., Stefanadis, C., Chrousos, G. P., & Tousoulis, D. (2018) · Metabolism · doi:10.1016/j.metabol.2018.10.002 · Cited by 93
“Publication bias was evaluated by the visual inspection of funnel plots, since the high inter-study heterogeneity precluded the safe interpretation of the available statistical tests [17].”
MethodA meta-analysis of biomarkers related to oxidative stress and nitric oxide pathway in migraine
Neri, M., Frustaci, A., Milić, M., Valdiglesias, V., Fini, M., Bonassi, S., & Barbanti, P. (2015) · Cephalalgia · doi:10.1177/0333102414564888 · Cited by 89
“No evidence of publication bias was found in any meta-analysis, although this measure has low reliability as fewer than 10 studies were evaluated (47).”
BackgroundIoannidou, E., Siempos, I. Ι., & Falagas, M. E. (2007) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkm385 · Cited by 89
“Klastersky/16 NA: 14/18 (78) versus 9/20 ( 45 ) NA: 14/18 (78) versus 9/20 (45) 1/14 versus NA 5/18 (28) versus 8/20 (40) 8/18 (44) versus 10/20 (50)”
InfluentialSun, Y., Li, T., & Gan, T. J. (2015) · Regional Anesthesia & Pain Medicine · doi:10.1097/aap.0000000000000273 · Cited by 89
Forest plots in reports of systematic reviews: a cross-sectional study reviewing current practice
Schriger, D. L., Altman, D. G., Vetter, J. A., Heafner, T., & Moher, D. (2010) · International Journal of Epidemiology · doi:10.1093/ije/dyp370 · Cited by 87
Zachariae, R., & O'Toole, M. S. (2015) · Psycho-Oncology · doi:10.1002/pon.3802 · Cited by 88
Systematic reviews and meta-analysis
Egger, M., Smith, G. D., & Sterne, J. A. C. (2009) · Oxford University Press eBooks · doi:10.1093/med/9780199218707.003.0039 · Cited by 90
Plante, J., Turgeon, A. F., Zarychanski, R., Lauzier, F., Vigneault, L., Moore, L., Boutin, A., & Fergusson, D. A. (2012) · BMJ · doi:10.1136/bmj.e5389 · Cited by 88
Moreno, S. G., Sutton, A. J., Thompson, J. R., Ades, A. E., Abrams, K. R., & Cooper, N. J. (2012) · Statistics in Medicine · doi:10.1002/sim.4488 · Cited by 90
“The use of a function of study precision as a covariate is commonly made to test for the presence of publication bias [27], and it has been seen to dominate other covariates when attempting to explain heterogeneity [26, 28–30].”
BackgroundKwok, C. S., Harrison, T. P., & Servant, C. (2013) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2012.09.005 · Cited by 87
Kite, T. A., Kurmani, S., Bountziouka, V., Cooper, N. J., Lock, S., Gale, C. P., Flather, M. D., Curzen, N. P., BANNING, A. P., McCann, G. P., et al. (2022) · European Heart Journal · doi:10.1093/eurheartj/ehac213 · Cited by 90
“Publication bias testing was inappropriate due to the small (,10) number of included studies.(33) Sensitivity analyses conducted via a leave-one-out meta-analysis did not change the statistical significance of the results.”
Tse, F., Yuan, Y., Moayyedi, P., Leontiadis, G. I., & Barkun, A. N. (2016) · Endoscopy · doi:10.1055/s-0042-119035 · Cited by 85
“Publication bias Application of funnel plot asymmetry tests to detect publication bias is not meaningful for this review because only 7 studies were included [22].”
Mathur, M. B., & VanderWeele, T. J. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1464 · Cited by 92
Weber, M., Schumacher, S., Hannig, W., Barth, J., Lotzin, A., Schäfer, I., Ehring, T. W., & Kleim, B. (2021) · Psychological Medicine · doi:10.1017/s003329172100163x · Cited by 99
“First, publication bias could not be tested due to small databases and considerable heterogeneity (Ioannidis & Trikalinos, 2007; Rothstein et al., 2005; Sterne et al., 2006; Terrin et al., 2003).”
BackgroundMethodWeththasinghe, P., Hansen, J. Ø., Mydland, L. T., & Øverland, M. (2021) · Reviews in Aquaculture · doi:10.1111/raq.12635 · Cited by 98
Alayo, Q. A., Fenster, M., Altayar, O., Glassner, K. L., Llano, E. M., Clark‐Snustad, K., Patel, A. A., Kwapisz, L., Yarur, A. J., Cohen, B. L., et al. (2022) · Crohn s & Colitis 360 · doi:10.1093/crocol/otac002 · Cited by 85
Habitual chocolate consumption and risk of cardiovascular disease among healthy men and women
Kwok, C. S., Boekholdt, S. M., Lentjes, M. A. H., Loke, Y. K., Luben, R., Yeong, J. K.-Y., Wareham, N. J., Myint, P. K., & Khaw, K. (2015) · Heart · doi:10.1136/heartjnl-2014-307050 · Cited by 87
Ioannidis, J. P. A., & Katritsis, D. G. (2007) · American Heart Journal · doi:10.1016/j.ahj.2007.07.049 · Cited by 86
Self-citationUnilateral versus bilateral upper limb exercise therapy after stroke: A systematic review
van Delden, A. E. Q., Peper, C. E., Beek, P. J., & Kwakkel, G. (2012) · Journal of Rehabilitation Medicine · doi:10.2340/16501977-0928 · Cited by 84
Eyawo, O., Brockman, G., Goldsmith, C. H., Hull, M., Lear, S. A., Bennett, M. T., Guillemi, S. A., Franco‐Villalobos, C., Adam, A., Mills, E. J., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-025874 · Cited by 83
“Although we assessed publication bias by visually inspecting and testing for funnel plot asymmetry,(47) its interpretation was limited by a lack of sufficient number of studies per meta-analysis.(48) 49 A p value <0.”
MethodCronström, A., Tengman, E., & Häger, C. K. (2021) · Sports Medicine · doi:10.1007/s40279-020-01424-3 · Cited by 89
Monaco, A. D., Pelargonio, G., Narducci, M. L., Manzoli, L., Boccia, S., Flacco, M. E., Capasso, L., Barone, L., Perna, F., Bencardino, G., et al. (2014) · EP Europace · doi:10.1093/europace/euu137 · Cited by 83
Kwok, C. S., Tiong, D., Pradhan, A., Andreou, A. Y., Nolan, J., Bertrand, O. F., Curzen, N. P., Urban, P. M., Myint, P. K., Zaman, A., et al. (2016) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2016.05.059 · Cited by 82
“Publication bias was assessed using funnel plots for an analysis with >10 studies and no evidence of substantial heterogeneity(31).”
MethodTreatment options for hirsutism: a systematic review and network meta-analysis
Barrionuevo, P., Nabhan, M., Altayar, O., Wang, Z., Erwin, P. J., Asi, N., Martin, K. A., & Murad, M. H. (2018) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2017-02052 · Cited by 82
Fan, H., Wu, F., Liu, J., Zeng, W., Zheng, S., Tian, H., Li, H., Yang, H., Wang, Z., Deng, Z., et al. (2021) · Annals of Translational Medicine · doi:10.21037/atm-20-4576 · Cited by 81
“Publication bias was evaluated using funnel plots, Begg’s tests, and Egger’s tests (38).”
Orzuna‐Orzuna, J. F., Dorantes-Iturbide, G., Lara-Bueno, A., Martínez, G. D. M., Miranda‐Romero, L. A., & Hernández-García, P. A. (2021) · Sustainability · doi:10.3390/su13137410 · Cited by 102
“10) and when the variable to be assessed is not reported in at least 10 studies [47].”
BackgroundYang, Y., Sánchez‐Tójar, A., O’Dea, R. E., Noble, D. W. A., Koricheva, J., Jennions, M. D., Parker, T. H., Lagisz, M., & Nakagawa, S. (2023) · BMC Biology · doi:10.1186/s12915-022-01485-y · Cited by 111
“Medical researchers found a similar percentage of meta-analyses showing small-study effects (7–18%) in a survey of 6873 meta-analyses (the large sample is because medical research has a bigger pool of metaanalyses to draw from and because that study extracted a much narrower scope of data from each meta-analysis than…”
BackgroundMethodFontana, A., Spadaro, S., Copetti, M., Spoto, B., Salvemini, L., Pizzini, P., Frittitta, L., Mallamaci, F., Pellegrini, F., Trischitta, V., et al. (2015) · PLoS ONE · doi:10.1371/journal.pone.0120419 · Cited by 81
“[24] recommendations, we did not perform tests for publication bias.”
MethodResultPang, B., Cheng, S., Sun, S., An, C., Liu, Z., Xue, F., & Liu, G. (2014) · Scientific Reports · doi:10.1038/srep06255 · Cited by 84
Farver-Vestergaard, I., Jacobsen, D., & Zachariae, R. H. (2014) · Psychotherapy and Psychosomatics · doi:10.1159/000367635 · Cited by 85
Are Biologics Efficacious in Atopic Dermatitis? A Systematic Review and Meta-Analysis
Snast, I., Reiter, O., Hodak, E., Friedland, R., Mimouni, D., & Leshem, Y. A. (2017) · American Journal of Clinical Dermatology · doi:10.1007/s40257-017-0324-7 · Cited by 81
A systematic review and meta-analysis of placebo versus no treatment for insomnia symptoms
Yeung, V., Sharpe, L., Glozier, N., Hackett, M. L., & Colagiuri, B. (2017) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2017.03.006 · Cited by 83
Han, M. A., Kim, J. H., & Song, H. S. (2018) · European Journal of Cancer Prevention · doi:10.1097/cej.0000000000000481 · Cited by 79
Immersive procedural training in virtual reality: A systematic literature review
Jongbloed, J., Chaker, R., & Lavoué, É. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105124 · Cited by 113
“While we find that several of these tests were significant for asymmetry, we cannot interpret these results with certainty because an asymmetrical funnel plot may be due to study heterogeneity rather than publication bias (Ioannidis & Trikalinos, 2007).”
Patient Expectancy and Post-chemotherapy Nausea: A Meta-analysis
Colagiuri, B., & Zachariae, R. H. (2010) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9186-4 · Cited by 79
“Publication bias, a widespread problem when conducting meta-analyses [ 40 ], was evaluated with the commonly used graphical funnel plot method [41], the Egger test, a widely used test for funnel plot assymetry [41], and calculation of failsafe numbers [42, 43].”
MethodMauro, M. F. F. P., Papelbaum, M., Brasil, M. A. A., Carneiro, J. R. I., Coutinho, E. S. F., Coutinho, W., & Appolinario, J. C. (2019) · Obesity Reviews · doi:10.1111/obr.12907 · Cited by 80
“Meta‐ regression analysis was tried to explore reasons of heterogeneity, but again, the small number of studies included compromised the appropriate use of thismethodological tool.(30) All analyseswere carried out using Stata 14.”
MethodOwen, S.-A., Lunt, M., Bowes, J., Hider, S., Bruce, I. N., Thomson, W., & Barton, A. C. (2011) · The Pharmacogenomics Journal · doi:10.1038/tpj.2011.42 · Cited by 77
Niglio, S. A., Jia, R., Ji, J., Ruder, S. F., Patel, V. G., Martini, A., Sfakianos, J. P., Marqueen, K. E., Waingankar, N., Mehrazin, R., et al. (2019) · European Urology · doi:10.1016/j.eururo.2019.05.037 · Cited by 77
Jurišić, V., Vuković, V., Obradović, J., Gulyaeva, L. F., Кушлинский, Н. Е., & Djordjević, N. (2020) · Journal of Oncology · doi:10.1155/2020/1973241 · Cited by 80
Efficacy of Antiplatelet Therapy in Secondary Prevention Following Lacunar Stroke
Kwok, C. S., Shoamanesh, A., Copley, H. C., Myint, P. K., Loke, Y. K., & Benavente, O. (2015) · Stroke · doi:10.1161/strokeaha.114.008422 · Cited by 78
Cronström, A., Tengman, E., & Häger, C. K. (2022) · Sports Medicine · doi:10.1007/s40279-022-01747-3 · Cited by 80
“If the meta-analysis included at least 10 studies and the corresponding I2 was ≤ 50%, funnel plots with trim and fill were used to evaluate any publication bias [27, 28].”
MethodStatic magnets for reducing pain: systematic review and meta-analysis of randomized trials
Pittler, M. H., Brown, E. M., & Ernst, E. K. (2007) · Canadian Medical Association Journal · doi:10.1503/cmaj.061344 · Cited by 76
Carbohydrate Ingestion during Endurance Exercise Improves Performance in Adults1,2
Temesi, J., Johnson, N. A., Raymond, J., Burdon, C. A., & O'Connor, H. T. (2011) · Journal of Nutrition · doi:10.3945/jn.110.137075 · Cited by 76
AGA Technical Review on Gastrointestinal Evaluation of Iron Deficiency Anemia
Rockey, D. C., Altayar, O., Falck-Ytter, Y. T., & Kalmaz, D. M. (2020) · Gastroenterology · doi:10.1053/j.gastro.2020.06.045 · Cited by 80
Functional MRS studies of GABA and glutamate/Glx – A systematic review and meta-analysis
Pasanta, D., He, J. L., Ford, T. C., Oeltzschner, G., Lythgoe, D. J., & Puts, N. A. J. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104940 · Cited by 83
Jin, Z., Liu, J., Li, R., Gan, T. J., He, Y., & Lin, J. (2020) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2020.109715 · Cited by 74
Katsigianni, M., Karageorgiou, V., Lambrinoudaki, I. V., & Siristatidis, C. S. (2019) · Molecular Psychiatry · doi:10.1038/s41380-019-0398-0 · Cited by 75
“If statistical significance was achieved, the Duvall and Tweedle trim-and-fill model was used as an approximation of the result if it was not affected by publication bias [36].”
BackgroundMethodBell, V., & Wade, D. M. (2020) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-020-01990-x · Cited by 77
“Inspection of the funnel plot suggested evidence of publication bias but the statistical tests of funnel plot asymmetry were not interpreted as the minimum criteria of 10 included studies was not met (Ioannidis and Trikalinos, 2007).”
BackgroundCleach, L. L., Trinquart, L., Do, G., Maruani, A., Lebrun‐Vignes, B., Ravaud, P., & Chosidow, O. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009036.pub2 · Cited by 70
The additional yield of combined nerve/muscle biopsy in vasculitic neuropathy
Vrancken, A. F. J. E., Gathier, C. S., Cats, E. A., Notermans, N. C., & Collins, M. P. (2010) · European Journal of Neurology · doi:10.1111/j.1468-1331.2010.03041.x · Cited by 72
“Funnel plots were used to assess bias by small-study effects [16].”
MethodWu, P., Kwok, C. S., Haththotuwa, R., Kotronias, R. A., Babu, A., Fryer, A. A., Myint, P. K., Chew-Graham, C. A., & Mamas, M. A. (2016) · Diabetologia · doi:10.1007/s00125-016-4098-x · Cited by 72
“We planned to conduct asymmetry testing for publication bias provided there were more than ten studies in the meta-analysis and statistical heterogeneity was less than 50% [16].”
MethodIlyas, A., Hübel, C., Stahl, D. R., Stadler, M., Ismail, K., Breen, G. D., Treasure, J. L., & Kan, C. (2018) · Molecular and Cellular Endocrinology · doi:10.1016/j.mce.2018.10.005 · Cited by 71
Wang, J., Vaddu, S., Bhumanapalli, S., Mishra, A., Applegate, T. J., Singh, M., & Thippareddi, H. (2023) · Poultry Science · doi:10.1016/j.psj.2023.102566 · Cited by 70
Patarčić, I., Gelemanović, A., Kirin, M., Kolčić, I., Τheodoratou, E., Baillie, J. K., de Jong, M. D., Rudan, I., Campbell, H. J., & Polašek, O. (2015) · Scientific Reports · doi:10.1038/srep16119 · Cited by 68
Interventions for necrotizing soft tissue infections in adults
Hua, C., Bosc, R., Sbidian, É., de Prost, N., Hughes, C., Jabre, P., Chosidow, O., & Cleach, L. L. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011680.pub2 · Cited by 69
Publication bias in psychological science: Comment on Ferguson and Brannick (2012).
Rothstein, H. R., & Bushman, B. J. (2012) · Psychological Methods · doi:10.1037/a0027128 · Cited by 73
Burton, B. K., Hjorthøj, C. R., Jepsen, J. R. M., Thorup, A. A. E., Nordentoft, M., & Plessen, K. J. (2015) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12479 · Cited by 68
“As no large and significant between-study heterogeneity was identified, we applied tests to evaluate potential publication bias (Ioannidis & Trikalinos, 2007).”
MethodInfluentialBlomberg, B. A., Moghbel, M. C., Saboury, B., Stanley, C. A., & Alavi, A. A. (2012) · Molecular Imaging and Biology · doi:10.1007/s11307-012-0572-0 · Cited by 70
“Application among metaanalysis with small number of studies (nG10) yields low statistical power [46].”
BackgroundJafari, T., Rostampour, N., Fallah, A. A., & Hesami, A. (2017) · Journal of Trace Elements in Medicine and Biology · doi:10.1016/j.jtemb.2017.09.002 · Cited by 68
“Publication bias was evaluated by visual inspection of Begg’s funnel plot asymmetry and Egger's test [23,24].”
MethodBlood Transfusion After Percutaneous Coronary Intervention and Risk of Subsequent Adverse Outcomes
Kwok, C. S., Sherwood, M. W., Watson, S. M., Nasir, S. B., Sperrin, M., Nolan, J., Kinnaird, T. D., Kiatchoosakun, S., Ludman, P., de Belder, M., et al. (2015) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2014.09.026 · Cited by 68
“Publication bias was assessed using funnel plots when there were >10 studies available in the meta-analysis and there was no evidence of substantial statistical heterogeneity (12).”
MethodThe Effects of Extensive Reading on English Vocabulary Learning: A Meta-analysis
Liu, J., & Zhang, J. (2018) · English Language Teaching · doi:10.5539/elt.v11n6p1 · Cited by 67
Sugar rush or sugar crash? A meta-analysis of carbohydrate effects on mood
Mantantzis, K., Schlaghecken, F., Sünram-Lea, S. I., & Maylor, E. A. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.03.016 · Cited by 67
Prepectoral Versus Subpectoral Implant-Based Breast Reconstruction
Li, Y., Xu, G., Yu, N., Huang, J., & Long, X. (2020) · Annals of Plastic Surgery · doi:10.1097/sap.0000000000002190 · Cited by 67
Jafari, T., Feizi, A., Askari, G., & Fallah, A. A. (2014) · Clinical Nutrition · doi:10.1016/j.clnu.2014.05.008 · Cited by 69
“The funnel plot, Begg and Mazumdar rank correlation test and Egger test were employed to assess the publication bias [29].”
MethodMazuquin, B., Wright, A. C., Russell, S., Monga, P., Selfe, J., & Richards, J. (2016) · British Journal of Sports Medicine · doi:10.1136/bjsports-2016-095963 · Cited by 68
“9”
Vasopressin for cardiac arrest: Meta-analysis of randomized controlled trials
Mentzelopoulos, S. D., Zakynthinos, S. G., Siempos, I. Ι., Malachias, S., Ulmer, H., & Wenzel, V. (2011) · Resuscitation · doi:10.1016/j.resuscitation.2011.07.015 · Cited by 66
Cronström, A., Creaby, M. W., & Ageberg, E. (2020) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-020-03552-3 · Cited by 74
Child physical abuse risk factors: A systematic review and a meta-analysis
Milner, J. S., Crouch, J. L., McCarthy, R. J., Ammar, J., Dominguez-Martinez, R., Thomas, C. L., & Jensen, A. P. (2022) · Aggression and Violent Behavior · doi:10.1016/j.avb.2022.101778 · Cited by 75
Chen, S., Roffey, D. M., Dion, C.-A. B., Arab, A., & Wai, E. K. (2015) · Clinical Journal of Pain · doi:10.1097/ajp.0000000000000218 · Cited by 68
Buawangpong, N., Teekachunhatean, S., & Koonrungsesomboon, N. (2020) · Pharmacology Research & Perspectives · doi:10.1002/prp2.644 · Cited by 66
Qorri, A., Gashi, S., & Krasławski, A. (2020) · Sustainable Development · doi:10.1002/sd.2140 · Cited by 65
Food insecurity and hypertension: A systematic review and meta-analysis
Beltrán, S., Pharel, M., Montgomery, C. T., López-Hinojosa, I. J., Arenas, D. J., & DeLisser, H. M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0241628 · Cited by 65
“As for publication bias, although no funnel asymmetry test was significant, it is important to note that the statistical power of these tests is small for the number of reviewed studies [27, 77, 78].”
BackgroundAndrew, B. J., Mullan, B. A., de Wit, J. B. F., Monds, L. A., Todd, J., & Kothe, E. J. (2016) · AIDS and Behavior · doi:10.1007/s10461-016-1314-0 · Cited by 73
“Unfortunately, upon completing these analyses, the substantial heterogeneity observed in each estimated relationship meant that formal statistical testing of publication bias could not be performed [66, 67].”
BackgroundBatchelor, R., Kumar, R., Gilmartin‐Thomas, J., Hopper, I., Kemp, W. W., & Liew, D. (2018) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.14955 · Cited by 65
Gu, L., Khadaroo, P. A., Chen, L., Li, X., Zhu, H., Zhong, X., Pan, J., & Chen, M. (2019) · Journal of Gastrointestinal Surgery · doi:10.1007/s11605-019-04227-8 · Cited by 64
“05 was considered statistically significant.(16)”
Childhood residential mobility and health outcomes: A meta-analysis
Şimşek, M., Costa, R., & de Valk, H. A. G. (2021) · Health & Place · doi:10.1016/j.healthplace.2021.102650 · Cited by 72
Burdon, C. A., Johnson, N. A., Chapman, P., & O'Connor, H. T. (2012) · International Journal of Sport Nutrition and Exercise Metabolism · doi:10.1123/ijsnem.22.3.199 · Cited by 65
“Publication bias was not assessed to avoid inappropriate interpretation based on analysis of too few studies (Ioannidis & Trikalinos, 2007).”
MethodVerschuère, B., Köbis, N., Bereby‐Meyer, Y., Rand, D. G., & Shalvi, S. (2018) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2018.04.005 · Cited by 63
“Finding substantial lie–truth differences in RTs, our findings support the cognitive theory of lying But our findings also show that, for the well-established RT index of lying, imposing additional load may in fact hinder truth-telling, to the extent that the differences between lying and…”
ResultRanda, H., Todberg, T., Skov, L., Larsen, L. P. S., & Zachariae, R. H. (2016) · Acta Dermato Venereologica · doi:10.2340/00015555-2600 · Cited by 63
Primer: strengths and weaknesses of meta-analysis
Finckh, A., & Tramèr, M. R. (2008) · Nature Clinical Practice Rheumatology · doi:10.1038/ncprheum0732 · Cited by 62
“Unfortunately, these tests are only exploratory and often misused to make inappropriately strong statements about the presence or not of publication bias in meta-analyses”
BackgroundTurner, L., Boutron, I., Hróbjartsson, A., Altman, D. G., & Moher, D. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-79 · Cited by 66
“An array of methods to test for publication bias and additional recommendations are now available [38,43,67-76], many of which have been evaluated [77-80].”
Chang, J., Liu, X., Wang, S., Zhang, Z., Wu, Z., Zhang, X., & Li, J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0105524 · Cited by 61
“Also, it is recommended that tests for funnel plot asymmetry should be used only when at least 10 studies are included in the meta-analysis [39].”
MethodKwok, C. S., Aslam, S., Kontopantelis, E., Myint, P. K., Zaman, M., Buchan, I. E., Loke, Y. K., & Mamas, M. A. (2015) · International Journal of Clinical Practice · doi:10.1111/ijcp.12646 · Cited by 61
Tse, F., Yuan, Y., Bukhari, M. A., Leontiadis, G. I., Moayyedi, P., & Barkun, A. N. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010571.pub2 · Cited by 60
Association of the ACTN3 R577X (rs1815739) polymorphism with elite power sports: A meta-analysis
Tharabenjasin, P., Pabalan, N., & Jarjanazi, H. (2019) · PLoS ONE · doi:10.1371/journal.pone.0217390 · Cited by 60
Metadata
- Authors
- Ioannidis, J. P. A., & Trikalinos, T. A.
- Affiliations
- Tufts University (US)
- Venue
- Canadian Medical Association Journal
- Publisher
- CMA Joule Inc.
- Published
- 2007-04-10
- Volume
- 176
- Issue
- 8
- Pages
- 1091-1096
- ISSN
- 0820-3946 (ISSN-L)
- 1488-2329
- PMC
- PMC1839799
- OpenAlex
- W2164093802
- Cited by
- 1,052
- References
- 40
- Citations by year
- 2026: 49 · 2025: 48 · 2024: 64 · 2023: 49 · 2022: 81 · 2021: 73 · 2020: 83 · 2019: 81 · 2018: 73 · 2017: 55 · 2016: 62 · 2015: 64 · 2014: 68 · 2013: 38 · 2012: 39
- FWCI
- 31.66
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Canadian Medical Association Journal (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, funnel plot asymmetry, meta-analysis, heterogeneity, Cochrane Database of Systematic Reviews, binary outcomes, test concordance
- MeSH terms
- Data Interpretation, Statistical, Humans, Periodicals as Topic, Meta-Analysis as Topic, Databases, Bibliographic, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
