The appropriateness of asymmetry tests for publication bias in meta-analyses: A large survey
Open accessAbstract
BACKGROUND: Statistical tests for funnel-plot asymmetry are common in meta-analyses. Inappropriate application can generate misleading inferences about publication bias. We aimed to measure, in a survey of meta-analyses, how frequently the application of these tests would be not meaningful or inappropriate. METHODS: We evaluated all meta-analyses of binary outcomes with é 3 studies in the Cochrane Database of Systematic Reviews (2003, issue 2). A separate, restricted analysis was confined to the largest meta-analysis in each of the review articles. In each meta-analysis, we assessed whether criteria to apply asymmetry tests were met: no significant heterogeneity, I2 < 50%, é 10 studies (with statistically significant results in at least 1) and ratio of the maximal to minimal variance across studies > 4. We performed a correlation and 2 regression asymmetry tests and evaluated their concordance. Finally, we sampled 60 meta-analyses from print journals in 2005 that cited use of the standard regression test. RESULTS: A total of 366 of 6873 (5%) and 98 of 846 meta-analyses (12%) in the wider and restricted Cochrane data set, respectively, would have qualified for use of asymmetry tests. Asymmetry test results were significant in 7%-18% of the meta-analyses. Concordance between the 3 tests was modest (estimated k 0.33-0.66). Of the 60 journal meta-analyses, 7 (12%) would qualify for asymmetry tests; all 11 claims for identification of publication bias were made in the face of large and significant heterogeneity. INTERPRETATION: Statistical conditions for employing asymmetry tests for publication bias are absent from most meta-analyses; yet, in medical journals these tests are performed often and interpreted erroneously.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Publication bias tests might be problematic for intranasal oxytocin studies due to small sample sizes resulting in insufficient statistical power and sample size variability for the tests to yield reliable and significant results.
“However, publication bias tests might be problematic for IN-OT studies due to small sample sizes resulting in insufficient statistical power and sample size variability for the tests to yield reliable and significant results (26).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [26]
The Abstract says asymmetry tests have low power when a meta-analysis includes few studies and are not meaningful when studies have similar precision. The Interpretation likewise identifies too few studies and too little difference between study sizes as problems. The paper does not examine intranasal oxytocin studies or establish that their sample sizes have these characteristics; it also distinguishes few studies from small samples within studies. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe small number of studies and significant heterogeneity precluded formal testing for publication bias.
“The small number of studies and presence of significant heterogeneity precluded formal testing for publication bias[35].”
From Daily Sitting Time and All-Cause Mortality: A Meta-Analysis (2013) · cites this paper as [35]
Of 60 meta-analyses in 24 articles published in 2005 in which formal assessments were reported, most were based on fewer than ten studies; most displayed statistically significant heterogeneity; and many reviewers misinterpreted the results of the tests employed.
“Of 60 meta-analyses in 24 articles published in 2005 in which formal assessments were reported, most were based on fewer than ten studies; most displayed statistically significant heterogeneity; and many reviewers misinterpreted the results of the tests employed [123].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [123]
Evidence that smaller studies had larger estimated effects may suggest possible bias, but misinterpretation of such data is common.
“Although evidence that smaller studies had larger estimated effects than large ones may suggest the possibility that the available evidence is biased, misinterpretation of such data is common [123].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [123]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · see citing works
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · see citing works
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763 · see citing works
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Self-citationCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,228 · see citing works
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779 · see citing works
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · see citing works
Quantitative Synthesis in Systematic Reviews
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-9-199711010-00008 · Cited by 2,517 · see citing works
Self-citationPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368 · see citing works
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282 · see citing works
A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints
Harbord, R. M., Egger, M., & Sterne, J. A. C. (2006) · Statistics in Medicine · doi:10.1002/sim.2380 · Cited by 2,267 · cited 2× · see citing works
“, the natural logarithm of the odds ratio divided by its standard error] against its precision [the inverse of the standard error](8)) and a new variant, a modified version of the regression test, which has a lower false-positive rate.(10) For all tests, statistical significance was claimed for p < 0.”
BackgroundMethodCompeting interests: none declared.
Hirsch, L. (2006) · Cited by 1,396 · see citing works
Publication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291 · see citing works
A comparison of methods to detect publication bias in meta‐analysis
Macaskill, P., Walter, S. D., & Irwig, L. (2001) · Statistics in Medicine · doi:10.1002/sim.698 · Cited by 1,286 · see citing works
Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejme048225 · Cited by 1,241 · see citing works
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818 · see citing works
Adjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809 · see citing works
Ioannidis, J. P. A. (1998) · JAMA · doi:10.1001/jama.279.4.281 · Cited by 632 · see citing works
Self-citationChan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613 · see citing works
Publication Bias in Meta‐Analysis
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch1 · Cited by 603 · see citing works
DeAngelis, C. D., Drazen, J. M., Frizelle, F., Haug, C. J., Hoey, J., Horton, R. C., Kotzin, S., Laine, C. M., Marušić, A., Overbeke, A. J. P. M., et al. (2004) · JAMA · doi:10.1001/jama.292.11.1363 · Cited by 579 · see citing works
Jadad, A. R., Cook, D. J., Jones, A., Klassen, T. P., Tugwell, P., Moher, M., & Moher, D. (1998) · JAMA · doi:10.1001/jama.280.3.278 · Cited by 534 · see citing works
Terrin, N. C., Schmid, C. H., & Lau, J. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.006 · Cited by 493 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2019) · CHANCE · doi:10.1080/09332480.2019.1579573 · Cited by 473 · see citing works
Self-citationOutcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Chan, A.-W. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041086 · Cited by 444 · see citing works
Publication Bias: The Problem That Won't Go Away
DICKERSIN, K., & MIN, Y. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26343.x · Cited by 427 · see citing works
Misleading funnel plot for detection of bias in meta-analysis
Tang, J.-L., & Liu, J. L. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00204-8 · Cited by 420 · see citing works
Clinical trial registration: a statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P., et al. (2004) · The Lancet · doi:10.1016/s0140-6736(04)17034-7 · Cited by 408 · see citing works
Ioannidis, J. P. A., & Trikalinos, T. A. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.10.019 · Cited by 326 · see citing works
Self-citationSelective Reporting Biases in Cancer Prognostic Factor Studies
Kyzas, P., Loizou, K. T., & Ioannidis, J. P. A. (2005) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/dji184 · Cited by 231 · see citing works
Self-citationClinical trial registration: a statement from the International Committee of Medical Journal Editors
DeAngelis, C. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041281 · Cited by 118 · see citing works
Shea, B., Moher, D., Graham, I. D., Pham, B., & Tugwell, P. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001008 · Cited by 109 · see citing works
Mallett, S. V. (2003) · BMJ Evidence-Based Medicine · doi:10.1136/ebm.8.4.100 · Cited by 71 · see citing works
Extreme between-study homogeneity in meta-analyses could offer useful insights
Ioannidis, J. P. A., Trikalinos, T. A., & Zintzaras, E. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.02.013 · Cited by 58 · see citing works
Self-citationBeyond Trial Registration: A Global Trial Bank for Clinical Trial Reporting
Sim, I., & Detmer, D. E. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020365 · Cited by 46 · see citing works
Assessment of publication bias in meta-analyses of cardiovascular diseases
Palma, S., & Delgado‐Rodríguez, M. (2005) · Journal of Epidemiology & Community Health · doi:10.1136/jech.2005.033027 · Cited by 45 · see citing works
Ioannidis, J. P. A. (2005) · doi:10.1002/0470870168.ch15 · Cited by 44 · see citing works
Self-citationSusan Mallett, D., & Mike Clarke, D. (2003) · ACP Journal Club · doi:10.7326/acpjc-2003-139-1-a11 · Cited by 37 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · see citing works
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648 · see citing works
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578 · see citing works
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349 · see citing works
Self-citationSterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709 · see citing works
“After reviewing the results of these studies, and based on theoretical considerations, we formulated recommendations on testing for funnel plot asymmetry (box 2). The appendix describes the proposed tests, explains the reasons that some were not recommended, and discusses funnel plots for intervention effects measured…”
MethodSelf-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Italian Journal of Public Health · doi:10.2427/5768 · Cited by 3,374 · see citing works
Self-citationSystematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,237 · see citing works
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · see citing works
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542 · see citing works
Undue reliance on I 2 in assessing heterogeneity may mislead
Rücker, G., Schwarzer, G., Carpenter, J. R., & Schumacher, M. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-79 · Cited by 1,287 · see citing works
Griesdale, D. E. G., de Souza, R. J., van Dam, R. M., Heyland, D. K., Cook, D. J., Malhotra, A., Dhaliwal, R., Henderson, W. R., Chittock, D. R., Finfer, S., et al. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.090206 · Cited by 1,096 · see citing works
“bias in our analysis, the tests we used may be unreliable in the presence of significant heterogeneity.(41) However, since neither peer reviewers nor we have been able to assess the methodologic quality of the unpublished trials, we decided to include only trials published either in full or as abstracts in indexed jou…”
MethodThe effect of testing versus restudy on retention: A meta-analytic review of the testing effect.
Rowland, C. A. (2014) · Psychological Bulletin · doi:10.1037/a0037559 · Cited by 1,066 · see citing works
Dissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039 · see citing works
Chapter 10: Addressing reporting biases
Egger, M., & Moher, D. (2011) · Cited by 977 · see citing works
Jonas, D. E., Amick, H. R., Feltner, C., Bobashev, G., Thomas, K., Wines, R., Kim, M. M., Shanahan, E., Gass, C. E., Rowe, C. J., et al. (2014) · JAMA · doi:10.1001/jama.2014.3628 · Cited by 923 · see citing works
Daily Sitting Time and All-Cause Mortality: A Meta-Analysis
Chau, J. Y., Grunseit, A. C., Chey, T., Stamatakis, E., Brown, W. J., Matthews, C. E., Bauman, A. E., & van der Ploeg, H. P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0080000 · Cited by 876 · see citing works
“Evidence of publication bias The small number of studies and presence of significant heterogeneity precluded formal testing for publication bias[35].”
BackgroundShi, L., & Lin, L. (2019) · Medicine · doi:10.1097/md.0000000000015987 · Cited by 818 · see citing works
“the funnel plot’s asymmetry is truly caused by publication bias or confounded by heterogeneity.([49]) More sophisticated statistical methods (e.”
MethodJakobsen, J. C., Wetterslev, J., Winkel, P., Lange, T., & Gluud, C. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-120 · Cited by 704 · see citing works
“If there are no trials showing statistically significant effects, then publication bias may not be a plausible explanation for funnel plot asymmetry [13,71].”
BackgroundZachariae, R., Lyby, M. S., Ritterband, L. M., & O'Toole, M. S. (2016) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2015.10.004 · Cited by 704 · see citing works
Interpretation of tests of heterogeneity and bias in meta-analysis
Ioannidis, J. P. A. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.00986.x · Cited by 696 · see citing works
Self-citationBeaudart, C., Buckinx, F., Rabenda, V., Gillain, S., Cavalier, E., Slomian, J., Petermans, J., Reginster, J.-Y., & Bruyère, O. (2014) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2014-1742 · Cited by 676 · see citing works
“Moreover, because of the limited number of studies included in the meta-analyses on muscle mass and muscle power and because of the high heterogeneity observed in the metaanalysis on muscle strength, we were unable to measure the potential publication bias by the Begg’s adjusted rank correlation and the Egger’s regres…”
MethodMeta-Analyst: software for meta-analysis of binary, continuous and diagnostic data
Wallace, B., Schmid, C. H., Lau, J., & Trikalinos, T. A. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-80 · Cited by 658 · see citing works
Self-citationFerguson, C. J., & Heene, M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459059 · Cited by 603 · see citing works
Probiotics for the prevention of pediatric antibiotic-associated diarrhea
Goldenberg, J. Z., Lytvyn, L., Steurich, J., Parkin, P. C., Mahant, S., & Johnston, B. C. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004827.pub4 · Cited by 625 · see citing works
Hooper, L., Kay, C., Abdelhamid, A., Kroon, P. A., Cohn, J. S., Rimm, E. B., & Cassidy, A. (2012) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.111.023457 · Cited by 610 · see citing works
Risk of Clostridium difficile Infection With Acid Suppressing Drugs and Antibiotics: Meta-Analysis
Kwok, C. S., Arthur, A. K., Anibueze, C. I., Singh, S., Cavallazzi, R., & Loke, Y. K. (2012) · The American Journal of Gastroenterology · doi:10.1038/ajg.2012.108 · Cited by 579 · see citing works
Long-term Glycemic Variability and Risk of Adverse Outcomes: A Systematic Review and Meta-analysis
Gorst, C., Kwok, C. S., Aslam, S., Buchan, I., Kontopantelis, E., Myint, P. K., Heatlie, G., Loke, Y., Rutter, M. K., & Mamas, M. A. (2015) · Diabetes Care · doi:10.2337/dc15-1188 · Cited by 530 · see citing works
“10 studies and no evidence of statistical heterogeneity in a particular metaanalysis (34).”
BackgroundA meta-analysis of predictors of offender treatment attrition and its relationship to recidivism.
Olver, M. E., Stockdale, K. C., & Wormith, J. S. (2011) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0022200 · Cited by 524 · see citing works
Detecting and avoiding likely false‐positive findings – a practical guide
Forstmeier, W., Wagenmakers, E., & Parker, T. H. (2017) · Biological Reviews · doi:10.1111/brv.12315 · Cited by 436 · see citing works
Serum and plasma BDNF levels in major depression: A replication study and meta-analyses
Bocchio-Chiavetto, L., Bagnardi, V., Zanardini, R., Molteni, R., Gabriela Nielsen, M., Placentino, A., Giovannini, C., Rillosi, L., Ventriglia, M., Riva, M. A., et al. (2010) · The World Journal of Biological Psychiatry · doi:10.3109/15622971003611319 · Cited by 435 · see citing works
“In the presence of heterogeneity, no attempt to evaluate publication bias was done (Ioannidis and Trikalinos 2007).”
BackgroundMeta-analysis of filaggrin polymorphisms in eczema and asthma: Robust risk factors in atopic disease
Rodríguez, E., Baurecht, H., Herberich, E., Wagenpfeil, S., Brown, S., Cordell, H. J., Irvine, A. D., & Weidinger, S. (2009) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2009.03.036 · Cited by 428 · see citing works
Empirical Comparison of Publication Bias Tests in Meta-Analysis
Lin, L., Chu, H., Murad, M. H., Hong, C., Qu, Z., Cole, S. R., & Chen, Y. (2018) · Journal of General Internal Medicine · doi:10.1007/s11606-018-4425-7 · Cited by 464 · see citing works
Tauber, N. M., O’Toole, M. S., Dinkel, A., Galica, J., Humphris, G., Lebel, S., Maheu, C., Ozakinci, G., Prins, J., Sharpe, L., et al. (2019) · Journal of Clinical Oncology · doi:10.1200/jco.19.00572 · Cited by 402 · see citing works
“Publication Bias Positive and negative findings are not equally likely to be published, and publication bias is a widespread problem when reviewing available evidence.(32) We evaluated publication bias using funnel plots and Egger’s test.”
MethodPicetti, R., Deeney, M., Pastorino, S., Miller, M. R., Shah, A. S. V., Leon, D. A., Dangour, A. D., & Green, R. (2022) · Environmental Research · doi:10.1016/j.envres.2022.112988 · Cited by 419 · see citing works
“…per cancer type and study design meant that we did not fulfil at least one of the four conditions for reliably using asymmetry tests for publication bias, i.e. ≥ 10 studies (Ioannidis and Trikalinos, 2007), or a minimum of 10 studies per moderator to run a meta-regression (Deeks et al., 2019).”
Wozniak, D. R., Lasserson, T. J., & Smith, I. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007736.pub2 · Cited by 380 · see citing works
Frederiksen, Y., Farver-Vestergaard, I., Skovgård, N. G., Ingerslev, H. J., & Zachariae, R. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-006592 · Cited by 380 · see citing works
Moreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-2 · Cited by 373 · see citing works
Oxidative stress-related biomarkers in autism: Systematic review and meta-analyses
Frustaci, A., Neri, M., Cesario, A., Adams, J. B., Domenici, E., Bernardina, B. D., & Bonassi, S. (2012) · Free Radical Biology and Medicine · doi:10.1016/j.freeradbiomed.2012.03.011 · Cited by 370 · see citing works
“Visual analysis of funnel plots and Egger’s test revealed no publication bias, but it must be taken into account the low reliability of these methods when the number of included studies is small [98].”
MethodGriesdale, D. E. G., Liu, D., McKinney, J., & Choi, P. T. (2011) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-011-9620-5 · Cited by 361 · see citing works
“8) Sun 2005(15) 75/24/1/0 59/26/15/0 94/100 (94%) 97/100 (97%) 46 (43-49) 30 (28-33) Teoh 2009(27) 78/21/1/0 58/37/5/0 91/100 (91%) 98/100 (98%) 31.”
Tierney, J. F., Vale, C. L., Riley, R. D., Smith, C. T., Stewart, L. A., Clarke, M. J., & Rovers, M. M. (2015) · PLoS Medicine · doi:10.1371/journal.pmed.1001855 · Cited by 361 · see citing works
“Funnel plots may also help indicate the potential impact of both known and unknown missing trials on the results [25] but would not be appropriate in a meta-analysis including less than ten trials [25] or few trials with statistically significant results or one in which heterogeneity has been observed [26].”
Sensitivity Analysis for Publication Bias in Meta-Analyses
Mathur, M. B., & VanderWeele, T. J. (2020) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/rssc.12440 · Cited by 354 · see citing works
“In practice, most meta-analyses are far too small to apply selection models; for example, the percentage of meta-analyses comprising fewer than 10 studies probably exceeds 50% in the Cochrane database (Ioannidis and Trikalinos, 2007) and in medical journals (Sterne et al., 2000).”
BackgroundPrevalence and factors associated with polypharmacy: a systematic review and meta-analysis
Delara, M., Murray, L., Jafari, B., Bahji, A., Goodarzi, Z., Kirkham, J. G., Chowdhury, M., & Seitz, D. P. (2022) · BMC Geriatrics · doi:10.1186/s12877-022-03279-x · Cited by 348 · see citing works
“Publication bias was assessed for the prevalence of polypharmacy using funnel plots and Egger’s test [9].”
MethodStatistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies
Walum, H., Waldman, I. D., & Young, L. J. (2016) · Biological Psychiatry · doi:10.1016/j.biopsych.2015.06.016 · Cited by 345 · see citing works
Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215052 · Cited by 345 · see citing works
“Homogeneous subsets were created, because publication bias methods have poor statistical properties if the true effect size is heterogeneous [5, 32, 50, 52].”
BackgroundMethodGannon, T. A., Olver, M. E., Mallion, J. S., & James, M. (2019) · Clinical Psychology Review · doi:10.1016/j.cpr.2019.101752 · Cited by 335 · see citing works
Bariatric surgery and cardiovascular disease: a systematic review and meta-analysis
van Veldhuisen, S. L., Gorter, T. M., van Woerden, G., de Boer, R. A., Rienstra, M., Hazebroek, E. J., & van Veldhuisen, D. J. (2022) · European Heart Journal · doi:10.1093/eurheartj/ehac071 · Cited by 308 · see citing works
“Publication bias was tested with funnel plot asymmetry and Egger’s regression test if a minimum of 10 studies was included in the analysis.(38,39) Inter-rater agreement for the quality assessment was tested using Cohen’s kappa coefficient.”
Köhler, C. A., Evangelou, E., Stubbs, B., Solmi, M., Veronese, N., Belbasis, L., Bortolato, B., Melo, M. C. A., Coelho, C. A., Fernandes, B. S., et al. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.05.020 · Cited by 302 · see citing works
Self-citationExcess Significance Bias in the Literature on Brain Volume Abnormalities
Ioannidis, J. P. A. (2011) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.28 · Cited by 299 · see citing works
Self-citationKotsifaki, R., Korakakis, V., King, E., Barbosa, O., Maree, D., Pantouveris, M., Bjerregaard, A., Luomajoki, J., Wilhelmsen, J., & Whiteley, R. (2023) · British Journal of Sports Medicine · doi:10.1136/bjsports-2022-106158 · Cited by 303 · see citing works
Evaluation of excess significance bias in animal studies of neurological diseases
Tsilidis, K. K., Panagiotou, O. A., Sena, E. S., Aretouli, E., Evangelou, E., Howells, D. W., Salman, R. A.-S., Macleod, M. R., & Ioannidis, J. P. A. (2013) · PLoS Biology · doi:10.1371/journal.pbio.1001609 · Cited by 295 · see citing works
“However, these methods may not be very sensitive or specific in the detection of such biases, especially when few studies are included in a meta-analysis [11–13].”
MethodSelf-citationTreatment-effect estimates adjusted for small-study effects via a limit meta-analysis
Rücker, G., Schwarzer, G., Carpenter, J. R., Binder, H., & Schumacher, M. C. (2010) · Biostatistics · doi:10.1093/biostatistics/kxq046 · Cited by 284 · see citing works
“However, when applied following a prespecified analysis protocol, with their limitations duly acknowledged, we argue that such concerns (Ioannidis and Trikalinos, 2007) are minimized and that testing is a useful aide to researchers in judging funnel plots.”
Atrial fibrillation and incidence of dementia
Kwok, C. S., Loke, Y. K., Hale, R., Potter, J. F., & Myint, P. K. (2011) · Neurology · doi:10.1212/wnl.0b013e31820f2e38 · Cited by 278 · see citing works
Effectiveness of disease-management programs for improving diabetes care: a meta-analysis
Pimouguet, C., Goff, M. L., Thiébaut, R., Jf, D., & Helmer, C. (2010) · Canadian Medical Association Journal · doi:10.1503/cmaj.091786 · Cited by 276 · see citing works
Loneliness in Psychosis: A Meta-analytical Review
da Rocha, B. M., Rhodes, S., Vasilopoulou, E., & Hutton, P. (2017) · Schizophrenia Bulletin · doi:10.1093/schbul/sbx036 · Cited by 286 · see citing works
PD-L1 and gastric cancer prognosis: A systematic review and meta-analysis
Gu, L., Chen, M., Guo, D., Zhu, H., Zhang, W., Pan, J., Zhong, X., Li, X., Qian, H., & Wang, X. (2017) · PLoS ONE · doi:10.1371/journal.pone.0182692 · Cited by 280 · see citing works
Kuester, A., Niemeyer, H., & Knaevelsrud, C. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.11.004 · Cited by 261 · see citing works
Publication bias: what is it? How do we measure it? How do we avoid it?
Song, F., Hooper, L., & Loke, Y. (2013) · Open Access Journal of Clinical Trials · doi:10.2147/oajct.s34419 · Cited by 261 · see citing works
MacArthur, G. J., Harrison, S., Caldwell, D. M., Hickman, M., & Campbell, R. (2016) · Addiction · doi:10.1111/add.13224 · Cited by 265 · see citing works
Marital status and risk of cardiovascular diseases: a systematic review and meta-analysis
Wong, C. W., Kwok, C. S., Narain, A., Gulati, M., Mihailidou, A. S., Wu, P., Alasnag, M. A., Myint, P. K., & Mamas, M. A. (2018) · Heart · doi:10.1136/heartjnl-2018-313005 · Cited by 271 · see citing works
“plots if there is no evidence of significant heterogeneity.[23] Further subgroup analyses were”
BackgroundContract farming for improving smallholder incomes: What can we learn from effectiveness studies?
Ton, G., Vellema, W., Desiere, S., Weituschat, C. S., & D’Haese, M. (2017) · World Development · doi:10.1016/j.worlddev.2017.11.015 · Cited by 283 · see citing works
Peery, A. F., Kelly, C. R., Kao, D. H., Vaughn, B. P., Lebwohl, B., Singh, S. S., Imdad, A., & Altayar, O. (2024) · Gastroenterology · doi:10.1053/j.gastro.2024.01.008 · Cited by 282 · see citing works
Kwok, C. S., Pradhan, A. K., Khan, M. A., Anderson, S. G., Keavney, B. D., Myint, P. K., Mamas, M. A., & Loke, Y. K. (2014) · International Journal of Cardiology · doi:10.1016/j.ijcard.2014.02.026 · Cited by 265 · see citing works
Patel, S. H., Wang, Z., Wong, W. W., Murad, M. H., Buckey, C. R., Mohammed, K. S., Alahdab, F., Altayar, O., Nabhan, M., Schild, S. E., et al. (2014) · The Lancet Oncology · doi:10.1016/s1470-2045(14)70268-2 · Cited by 263 · see citing works
Invited review: Use of meta-analysis in animal health and reproduction: Methods and applications
Lean, I. J., Rabiee, A. R., Duffield, T. F., & Dohoo, I. R. (2009) · Journal of Dairy Science · doi:10.3168/jds.2009-2140 · Cited by 260 · see citing works
“If there are no statistically significant studies, then publication bias may not be a plausible explanation for funnel plot asymmetry (Ioannidis and Trikalinos, 2007).”
BackgroundProbiotics for the prevention of pediatric antibiotic-associated diarrhea
Johnston, B. C., Goldenberg, J. Z., Vandvik, P. O., Sun, X., & Guyatt, G. H. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004827.pub3 · Cited by 258 · see citing works
Computerized advice on drug dosage to improve prescribing practice
Durieux, P., Trinquart, L., Colombet, I., Niès, J., Walton, R. T., Rajeswaran, A., Rège-Walther, M., Harvey, E., & Burnand, B. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002894.pub2 · Cited by 251 · see citing works
Kopterides, P., Siempos, I. Ι., Tsangaris, I., Tsantes, A. Ε., & Armaganidis, A. (2010) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181f17bf9 · Cited by 243 · see citing works
“As a result of the small number of the included trials, detection of publication bias (29) and metaregression analysis (30) to explore sources of heterogeneity among studies were not performed.”
BackgroundCarter, E. C., & McCullough, M. E. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00823 · Cited by 381 · see citing works
Revisiting the serotonin–aggression relation in humans: A meta-analysis.
Duke, A. A., Bègue, L., Bell, R., & Eisenlohr-Moul, T. A. (2013) · Psychological Bulletin · doi:10.1037/a0031544 · Cited by 243 · see citing works
“All four criteria were met for statistically testing funnel plot asymmetry (see Ioannidis & Trikalinos, 2007).”
Prophylactic pancreatic stent placement and post-ERCP pancreatitis: an updated meta-analysis
Mazaki, T., Mado, K., Masuda, H., & Shiono, M. (2013) · Journal of Gastroenterology · doi:10.1007/s00535-013-0806-1 · Cited by 236 · see citing works
“10 was considered to be significant [38].”
Narain, A., Kwok, C. S., & Mamas, M. A. (2016) · International Journal of Clinical Practice · doi:10.1111/ijcp.12841 · Cited by 223 · see citing works
Nikolopoulos, G. K., Paraskevis, D. N., Hatzitheodorou, E., Moschidis, Z., Sypsa, V., Zavitsanos, X., Kalapothaki, V., & Hatzakis, A. E. (2009) · Clinical Infectious Diseases · doi:10.1086/599110 · Cited by 230 · see citing works
“The appropriateness of the asymmetry tests was checked using criteria reported elsewhere [23].”
MethodMeta‐research: The art of getting it wrong
Ioannidis, J. P. A. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.19 · Cited by 231 · see citing works
“2010, 1 169--184 1 7 5 A number of tests have been developed by meta-analysts to detect and exclude publication bias, but I am afraid that they are used beyond their real capacity (Ioannidis, 2008a; Ioannidis and Trikalinos, 2007b; Lau et al., 2006).”
BackgroundInfluentialSelf-citationCatalogue of bias: publication bias
DeVito, N., & Goldacre, B. M. (2018) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2018-111107 · Cited by 226 · see citing works
Johannsen, M., Damholdt, M. F., Zachariae, R., Lundorff, M., Farver-Vestergaard, I., & O'Connor, M. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.04.065 · Cited by 214 · see citing works
How evidence‐based medicine is failing due to biased trials and selective publication
Every‐Palmer, S., & Howick, J. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12147 · Cited by 228 · see citing works
“Numerous methods are available, but none can identify or rule out selection bias entirely [54,58–61].”
MethodEfficacy and Safety of Finasteride Therapy for Androgenetic Alopecia
Mella, J. M., Perret, M. C., Manzotti, M., Catalano, H. N., & Guyatt, G. H. (2010) · Archives of Dermatology · doi:10.1001/archdermatol.2010.256 · Cited by 221 · see citing works
Methods for meta-analysis in genetic association studies: a review of their potential and pitfalls
Kavvoura, F. K., & Ioannidis, J. P. A. (2007) · Human Genetics · doi:10.1007/s00439-007-0445-9 · Cited by 205 · see citing works
“Unfortunately, in most cases these tests are used either inappropriately or meaninglessly (Ioannidis and Trikalinos 2007a).”
BackgroundSelf-citationRisk factors for severe perineal lacerations during childbirth
Pergialiotis, V., Vlachos, D., Πρωτόπαπας, Α., Pappa, K. I., & Vlachos, G. D. (2014) · International Journal of Gynecology & Obstetrics · doi:10.1016/j.ijgo.2013.09.034 · Cited by 211 · see citing works
“For indices that were evaluated in more than 10 studies, publication bias was assessed with the Egger test [8], using Comprehensive MetaAnalysis version 2 (Biostat, Englewood, NJ, USA).”
MethodWu, B., Gu, D.-Z., Yu, J., Yang, J., & Shen, W. (2020) · Infection Genetics and Evolution · doi:10.1016/j.meegid.2020.104485 · Cited by 209 · see citing works
Siempos, I. I., Ntaidou, T. K., & Falagas, M. E. (2010) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181c8fe4b · Cited by 204 · see citing works
“Given that the number of included trials was 10, detection of publication bias was considered meaningless (12).”
BackgroundSchoenleber, S. J., Kurtz, D. M., Talwalkar, J. A., Roberts, L. R., & Gores, G. J. (2009) · British Journal of Cancer · doi:10.1038/sj.bjc.6605017 · Cited by 207 · see citing works
“We also focused on using one statistical approach, as performing multiple tests can generate discordant results that confuse data interpretation (Ioannidis and Trikalinos, 2007).”
ResultKwok, C. S., Umar, S., Myint, P. K., Mamas, M. A., & Loke, Y. K. (2014) · International Journal of Cardiology · doi:10.1016/j.ijcard.2014.07.080 · Cited by 205 · see citing works
“If there were N10 studies available in themeta-analysis, with no evidence of substantial statistical heterogeneity, we aimed to generate funnel plots to assess the possibility of publication bias [15].”
MethodWu, P., Gulati, M., Kwok, C. S., Wong, C. W., Narain, A., O'Brien, S., Chew‐Graham, C. A., Verma, G., Kadam, U. T., & Mamas, M. A. (2018) · Journal of the American Heart Association · doi:10.1161/jaha.117.007809 · Cited by 204 · see citing works
“In the case of an analysis where there are more than 10 studies and little evidence of heterogeneity, we planned to perform funnel plots to assess for publication bias.(40) Sensitivity analysis was performed to consider the follow-up duration of the studies (<10, 10–30, and >30 years), gestation (<32 weeks versus 32–3…”
MethodVitamin C for preventing and treating pneumonia
Hemilä, H., & Louhiala, P. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005532.pub3 · Cited by 203 · see citing works
Meta-analysis: Combination Endoscopic and Drug Therapy to Prevent Variceal Rebleeding in Cirrhosis
González, R., Zamora, J., Gómez‐Camarero, J., Molinero, L.-M., Bañares, R., & Albillos, A. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-2-200807150-00007 · Cited by 203 · see citing works
Katritsis, D. G., Siontis, G. C. M., Kastrati, A., Hof, A. W. J. V. ’., Neumann, F.-J., Siontis, K. C. M., & Ioannidis, J. P. A. (2010) · European Heart Journal · doi:10.1093/eurheartj/ehq276 · Cited by 201 · see citing works
“We did not perform funnel plot asymmetry tests since they are inappropriate when only four trials are available.(16)”
MethodSelf-citationReview and meta-analysis of ab-interno trabeculectomy outcomes
Kaplowitz, K. B., Bussel, I. I., Honkanen, R., Schuman, J. S., & Loewen, N. A. (2016) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2015-307131 · Cited by 197 · see citing works
Tse, F., & Yuan, Y. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009779.pub2 · Cited by 195 · see citing works
A meta-analysis of factors related to recycling
Geiger, J. L., Steg, L., van der Werff, E., & Ünal, A. B. (2019) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2019.05.004 · Cited by 199 · see citing works
Optimum duration of regimens for Helicobacter pylori eradication
Yuan, Y., Ford, A. C., Khan, K. J., Gisbert, J. P., Forman, D., Leontiadis, G. I., Tse, F., Calvet, X., Fallone, C. A., Fischbach, L. A., et al. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008337.pub2 · Cited by 194 · see citing works
Generalized Joint Hypermobility and Risk of Lower Limb Joint Injury During Sport
Pacey, V., Nicholson, L., Adams, R., Munn, J., & Munns, C. (2010) · American Journal of Sports Medicine · doi:10.1177/0363546510364838 · Cited by 192 · see citing works
The Efficacy and Safety of Heparin in Patients With Sepsis
Zarychanski, R., Abou-Setta, A. M., Kanji, S., Turgeon, A. F., Kumar, A., Houston, D. S., Rimmer, E. K., Houston, B. L., McIntyre, L., Fox‐Robichaud, A., et al. (2014) · Critical Care Medicine · doi:10.1097/ccm.0000000000000763 · Cited by 188 · see citing works
Regnaux, J.-P., Lefevre-Colau, M.-M., Trinquart, L., Nguyen, C., Boutron, I., Brosseau, L., & Ravaud, P. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010203.pub2 · Cited by 184 · see citing works
Aid and Growth: What Meta-Analysis Reveals
Mekasha, T. J., & Tarp, F. (2013) · The Journal of Development Studies · doi:10.1080/00220388.2012.709621 · Cited by 195 · see citing works
Cognitive behavioural therapy for psychosis prevention: a systematic review and meta-analysis
Hutton, P., & Taylor, P. J. (2014) · Psychological Medicine · doi:10.1017/s0033291713000354 · Cited by 184 · see citing works
Polyzos, N. P., Polyzos, I. P., Zavos, A., Valachis, A., Mauri, D. N., Papanikolaou, E. G. E., Tzioras, S., Weber, D., & Messinis, I. (2010) · BMJ · doi:10.1136/bmj.c7017 · Cited by 186 · see citing works
“Thus, early trials that did not favour treatment of periodontal with scaling and root planing during pregnancy may have been left unpublished, resulting in the accumulation of small trials with significant results in favour of treatment. According to the funnel plot constructed, publication bias is highly likely to ex…”
ResultMoreno, S. G., Sutton, A. J., Turner, E. H., Abrams, K. R., Cooper, N. J., Palmer, T. M., & Ades, A. E. (2009) · BMJ · doi:10.1136/bmj.b2981 · Cited by 184 · see citing works
Meta-analysis using effect size distributions of only statistically significant studies.
van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015) · Psychological Methods · doi:10.1037/met0000025 · Cited by 265 · see citing works
Pharmacologic Interventions for Painful Diabetic Neuropathy
Griebeler, M. L., Morey‐Vargas, O. L., Brito, J. P., Τσάπας, Α., Wang, Z., Leon, B. G. C., Phung, O. J., Montori, V. M., & Murad, M. H. (2014) · Annals of Internal Medicine · doi:10.7326/m14-0511 · Cited by 184 · see citing works
“We standardized the pain scales used in studies by estimating SMDs using the Cohen d method and 95% credible intervals (CrIs) (10).”
BackgroundMethod
- Authors
- Ioannidis, J. P. A., & Trikalinos, T. A.
- Affiliations
- Tufts University (US)
- Venue
- Canadian Medical Association Journal
- Publisher
- CMA Joule Inc.
- Published
- 2007-04-10
- Volume
- 176
- Issue
- 8
- Pages
- 1091-1096
- ISSN
- 0820-3946 (ISSN-L)
- 1488-2329
- PMC
- PMC1839799
- OpenAlex
- W2164093802
- Cited by
- 1,052
- References
- 40
- Citations by year
- 2026: 49 · 2025: 48 · 2024: 64 · 2023: 49 · 2022: 81 · 2021: 73 · 2020: 83 · 2019: 81 · 2018: 73 · 2017: 55 · 2016: 62 · 2015: 64 · 2014: 68 · 2013: 38 · 2012: 39
- FWCI
- 31.66
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Canadian Medical Association Journal (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Keywords
- publication bias, funnel plot asymmetry, meta-analysis, heterogeneity, Cochrane Database of Systematic Reviews, binary outcomes, test concordance
- MeSH terms
- Data Interpretation, Statistical, Humans, Periodicals as Topic, Meta-Analysis as Topic, Databases, Bibliographic, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
