Interpretation of tests of heterogeneity and bias in meta-analysis
Abstract
Statistical tests of heterogeneity and bias, in particular publication bias, are very popular in meta-analyses. These tests use statistical approaches whose limitations are often not recognized. Moreover, it is often implied with inappropriate confidence that these tests can provide reliable answers to questions that in essence are not of statistical nature. Statistical heterogeneity is only a correlate of clinical and pragmatic heterogeneity and the correlation may sometimes be weak. Similarly, statistical signals may hint to bias, but seen in isolation they cannot fully prove or disprove bias in general, let alone specific causes of bias, such as publication bias in particular. Both false-positive and false-negative signals of heterogeneity and bias can be common and their prevalence may be anticipated based on some rational considerations. Here I discuss the major common challenges and flaws that emerge in using and interpreting statistical tests of heterogeneity and bias in meta-analyses. I discuss misinterpretations that can occur at the level of statistical inference, clinical/pragmatic inference and specific cause attribution. Suggestions are made on how to avoid these flaws, use these tests properly and learn from them.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These limitations are common in meta-analyses of cross-sectional studies.
“These limitations are common in meta-analyses of cross-sectional studies (Ioannidis, 2008).”
From Decision making capacity for treatment in psychiatric inpatients: a systematic review and meta-analysis (2024) · cites this paper as Ioannidis, 2008
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · see citing works
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · see citing works
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763 · see citing works
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955 · see citing works
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Self-citationThe combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380 · see citing works
Assessing heterogeneity in meta-analysis: Q statistic or I² index?
Huedo-Medina, T. B., Sánchez-Meca, J., Marín-Martínez, F., & Botella, J. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.2.193 · Cited by 4,256 · see citing works
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779 · see citing works
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · see citing works
How should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026 · see citing works
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955 · see citing works
Quantitative Synthesis in Systematic Reviews
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-9-199711010-00008 · Cited by 2,517 · see citing works
Self-citationPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368 · see citing works
Comparison of two methods to detect publication bias in meta-analysis.
Peters, J. L. (2006) · JAMA · doi:10.1001/jama.295.6.676 · Cited by 2,278 · see citing works
A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints
Harbord, R. M., Egger, M., & Sterne, J. A. C. (2006) · Statistics in Medicine · doi:10.1002/sim.2380 · Cited by 2,267 · see citing works
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · see citing works
(2000) · Queen Mary Research Online (Queen Mary University of London) · Cited by 1,795 · see citing works
The case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766 · see citing works
Self-citationPublication bias in meta-analysis: Prevention, assessment and adjustments
Viechtbauer, W. (2007) · Psychometrika · doi:10.1007/s11336-006-1450-y · Cited by 1,507 · see citing works
Statistics in Medicine — Reporting of Subgroup Analyses in Clinical Trials
Wang, R., Lagakos, S. W., Ware, J. H., Hunter, D. J., & Drazen, J. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmsr077003 · Cited by 1,445 · see citing works
The Existence of Publication Bias and Risk Factors for Its Occurrence
Dickersin, K. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100097014 · Cited by 1,417 · see citing works
A comparison of methods to detect publication bias in meta‐analysis
Macaskill, P., Walter, S. D., & Irwig, L. (2001) · Statistics in Medicine · doi:10.1002/sim.698 · Cited by 1,286 · see citing works
Controlling the risk of spurious findings from meta-regression
Higgins, J. P. T., & Thompson, S. G. (2004) · Statistics in Medicine · doi:10.1002/sim.1752 · Cited by 1,267 · see citing works
Uncertainty in heterogeneity estimates in meta-analyses
Ioannidis, J. P. A., Patsopoulos, N. A., & Evangelou, E. (2007) · BMJ · doi:10.1136/bmj.39343.408449.80 · Cited by 1,254 · see citing works
Self-citationPatsopoulos, N. A., Εvangelou, Ε., & Ioannidis, J. P. A. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dyn065 · Cited by 1,147 · see citing works
Self-citationThe appropriateness of asymmetry tests for publication bias in meta-analyses: A large survey
Ioannidis, J. P. A., & Trikalinos, T. A. (2007) · Canadian Medical Association Journal · doi:10.1503/cmaj.060410 · Cited by 1,052 · see citing works
Self-citationA consumer's guide to subgroup analyses
Oxman, A. D., & Guyatt, G. H. (1992) · Annals of Internal Medicine · doi:10.7326/0003-4819-116-1-78 · Cited by 914 · see citing works
BAUJAT, B., Mahé, C., Pignon, J., & Hill, C. (2002) · Statistics in Medicine · doi:10.1002/sim.1221 · Cited by 762 · see citing works
An exploratory test for an excess of significant findings
Ioannidis, J. P., & Trikalinos, T. A. (2007) · Clinical Trials · doi:10.1177/1740774507079441 · Cited by 751 · see citing works
Self-citationA note on graphical presentation of estimated odds ratios from several clinical trials
Galbraith, R. F. (1988) · Statistics in Medicine · doi:10.1002/sim.4780070807 · Cited by 663 · see citing works
Summing up evidence: one answer is not always enough
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1998) · The Lancet · doi:10.1016/s0140-6736(97)08468-7 · Cited by 646 · see citing works
Self-citationBias in clinical intervention research.
Gluud, L. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj069 · Cited by 636 · see citing works
Chan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613 · see citing works
Simmonds, M. C., Higgins, J. P. T., Stewart, L. A., Tierney, J. F., Clarke, M. J., & Thompson, S. G. (2005) · Clinical Trials · doi:10.1191/1740774505cn087oa · Cited by 557 · see citing works
Limitations of Applying Summary Results of Clinical Trials to Individual Patients
Kent, D. M., & Hayward, R. A. (2007) · JAMA · doi:10.1001/jama.298.10.1209 · Cited by 502 · see citing works
Relative Citation Impact of Various Study Designs in the Health Sciences
Patsopoulos, N. A. (2005) · JAMA · doi:10.1001/jama.293.19.2362 · Cited by 498 · see citing works
Terrin, N. C., Schmid, C. H., & Lau, J. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.006 · Cited by 493 · see citing works
Misleading funnel plot for detection of bias in meta-analysis
Tang, J.-L., & Liu, J. L. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00204-8 · Cited by 420 · see citing works
Lambert, P. C., Sutton, A. J., Abrams, K. R., & Jones, D. R. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00414-0 · Cited by 396 · see citing works
Cumulative meta-analysis of clinical trials builds evidence for exemplary medical care
Lau, J., Schmid, C. H., & Chalmers, T. C. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00106-z · Cited by 358 · see citing works
Schmid, C. H., Stark, P. C., Berlin, J. A., Landais, P., & Lau, J. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.12.001 · Cited by 245 · see citing works
Evolution of treatment effects over time: Empirical insight from recursive cumulative metaanalyses
Ioannidis, J. P. A., & Lau, J. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.98.3.831 · Cited by 190 · see citing works
Self-citationClaims of Sex Differences: An Empirical Assessment in Genetic Associations
Patsopoulos, N. A., Tatsioni, A., & Ioannidis, J. P. A. (2007) · JAMA · doi:10.1001/jama.298.8.880 · Cited by 150 · see citing works
Self-citationA simulation study comparing properties of heterogeneity measures in meta‐analyses
Mittlböck, M., & Heinzl, H. (2006) · Statistics in Medicine · doi:10.1002/sim.2692 · Cited by 143 · see citing works
Commentary: Meta-analysis of Individual Participants' Data in Genetic Epidemiology
Ioannidis, J. P. A. (2002) · American Journal of Epidemiology · doi:10.1093/aje/kwf031 · Cited by 117 · see citing works
Self-citationSelection in Reported Epidemiological Risks: An Empirical Assessment
Kavvoura, F. K., Liberopoulos, G., & Ioannidis, J. P. A. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040079 · Cited by 114 · see citing works
Self-citationAN ALTERNATIVE TO THE USE OF TWO-SIDED TESTS IN CLINICAL TRIALS
Dunnett, C. W., & Gent, M. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960830)15:16 · Cited by 110 · see citing works
The impact of high-risk patients on the results of clinical trials
Ioannidis, J. P. A., & Lau, J. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00149-2 · Cited by 108 · see citing works
Self-citationClinical trial registration: looking back and moving ahead
Laine, C. M., Horton, R. C., DeAngelis, C. D., Drazen, J. M., Frizelle, F., Godlee, F., Haug, C. J., Hébert, P. C., Kotzin, S., Marušić, A., et al. (2007) · The Lancet · doi:10.1016/s0140-6736(07)60894-0 · Cited by 93 · see citing works
Schwarzer, G., Antes, G., & Schumacher, M. (2002) · Statistics in Medicine · doi:10.1002/sim.1224 · Cited by 91 · see citing works
Recursive Cumulative Meta-analysis
Ioannidis, J. P. A., Contopoulos‐Ioannidis, D. G., & Lau, J. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(98)00159-0 · Cited by 84 · see citing works
Self-citationWhen Should Potentially False Research Findings Be Considered Acceptable?
Djulbegović, B., & Hozo, I. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040026 · Cited by 77 · see citing works
loannidis, J. P. A., & Lau, J. (1998) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009590 · Cited by 72 · see citing works
Extreme between-study homogeneity in meta-analyses could offer useful insights
Ioannidis, J. P. A., Trikalinos, T. A., & Zintzaras, E. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.02.013 · Cited by 58 · see citing works
Self-citationIoannidis, J. P. A., Cappelleri, J. C., Sacks, H. S., & Lau, J. (1997) · Controlled Clinical Trials · doi:10.1016/s0197-2456(97)00097-4 · Cited by 43 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709 · see citing works
Self-citationGloy, V. L., Briel, M., Bhatt, D. L., Kashyap, S. R., Schauer, P. R., Mingrone, G., Bucher, H. C., & Nordmann, A. J. (2013) · BMJ · doi:10.1136/bmj.f5934 · Cited by 1,351 · see citing works
Sonuga-Barke, E. J. S., Brandeis, D., Cortese, S., Daley, D., Ferrin, M., Holtmann, M., Stevenson, J., Danckaerts, M., van der Oord, S., Döpfner, M., et al. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2012.12070991 · Cited by 1,167 · see citing works
“Given the relatively small number of methodologically sound studies, the field is not yet mature enough for the investigation of publication bias using funnel plots—the interpretation of which, moreover, is equivocal when based on a small number of studies (29).”
BackgroundA systematic review of personality trait change through intervention.
Roberts, B. W., Luo, J., Briley, D. A., Chow, P. I., Su, R., & Hill, P. L. (2017) · Psychological Bulletin · doi:10.1037/bul0000088 · Cited by 1,047 · see citing works
“Most critically, Ioannidis (2008) argued that when effect sizes are even moderately heterogeneous, regression-based tests are “inappropriate, meaningless, or both” (p. 954).”
BackgroundZachariae, R., Lyby, M. S., Ritterband, L. M., & O'Toole, M. S. (2016) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2015.10.004 · Cited by 704 · see citing works
Probiotics for the prevention of pediatric antibiotic-associated diarrhea
Goldenberg, J. Z., Lytvyn, L., Steurich, J., Parkin, P. C., Mahant, S., & Johnston, B. C. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004827.pub4 · Cited by 625 · see citing works
Taylor, B., Irving, H. M., Kanteres, F., Room, R., Borges, G., Cherpitel, C., Greenfield, T., & Rehm, J. (2010) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2010.02.011 · Cited by 509 · see citing works
Global epidemiology of Duchenne muscular dystrophy: an updated systematic review and meta-analysis
Crisafulli, S., Sultana, J., Fontana, A., Salvo, F., Messina, S., & Trifirò, G. (2020) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-020-01430-8 · Cited by 505 · see citing works
“In response to this shortcoming, I(2) estimates were also accompanied by their associated 95% CI [70, 71] because imprecise or biased estimates of heterogeneity can have serious consequences: for instance its overestimation may trigger inappropriate exploration of the cause(s) of heterogeneity.”
BackgroundBedaso, A., Adams, J., Peng, W., & Sibbritt, D. W. (2021) · Reproductive Health · doi:10.1186/s12978-021-01209-5 · Cited by 549 · see citing works
“Publication bias, heterogeneity, and subgroup analysis Possible publication bias was assessed through inspection of the funnel plot and Egger’s regression tests [53, 54].”
MethodMorris, M. C., Compas, B. E., & Garber, J. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.02.002 · Cited by 466 · see citing works
Patra, J., Bakker, R., Irving, H., Jaddoe, V., Malini, S., & Rehm, J. (2011) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.2011.03050.x · Cited by 555 · see citing works
Lopez-Duran, N. L., Kovacs, M., & George, C. J. (2009) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2009.03.016 · Cited by 425 · see citing works
“…factors that explain differences between studies even if heterogeneity was not observed, as estimates of heterogeneity have some limitations (Ioannidis, 2008), and significant moderating factors may be present even when the variability between studies is not large (see Hall and Rosenthal,…”
BackgroundEvolution of Heterogeneity (I2) Estimates and Their 95% Confidence Intervals in Large Meta-Analyses
Thorlund, K., Imberger, G. L., Johnston, B. C., Walsh, M. W., Awad, T., Thabane, L., Gluud, C. N., Devereaux, P. J., & Wetterslev, J. (2012) · PLoS ONE · doi:10.1371/journal.pone.0039471 · Cited by 432 · see citing works
“In response to the above identified shortcomings, it has been proposed that reported I(2) estimates should be accompanied by their associated 95% confidence interval (CI) [6,12].”
BackgroundZhang, L., Rana, I., Shaffer, R. M., Taioli, E., & Sheppard, L. A. (2019) · Mutation Research/Reviews in Mutation Research · doi:10.1016/j.mrrev.2019.02.001 · Cited by 381 · see citing works
Meta-assessment of bias in science
Fanelli, D., Costas, R., & Ioannidis, J. P. A. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1618569114 · Cited by 364 · see citing works
“Moreover, small-study effects may result not necessarily from QRP but also from legitimate choices made in designing small studies (17, 36).”
BackgroundSelf-citationZymperdikas, V. F., Koretsi, V., Papageorgiou, S. N., & Papadopoulos, M. A. A. P. (2015) · European Journal of Orthodontics · doi:10.1093/ejo/cjv034 · Cited by 306 · see citing works
“All P values were 2-sided with α = 5%, except for the test of between-studies or betweensubgroups heterogeneity (α = 10%) (55).”
MethodTreatment-effect estimates adjusted for small-study effects via a limit meta-analysis
Rücker, G., Schwarzer, G., Carpenter, J. R., Binder, H., & Schumacher, M. C. (2010) · Biostatistics · doi:10.1093/biostatistics/kxq046 · Cited by 284 · see citing works
“We acknowledge that some authors take a censorious attitude to such tests, believing them misleading (Lau and others, 2006; Terrin and others, 2003; Tang and Liu, 2000), stigmatizing them as “pseudo tests” (Ioannidis, 2008), and questioning whether funnel plots are a suited means at all for judging small-study effects…”
BackgroundEffect of Hospital Volume on Surgical Outcomes After Pancreaticoduodenectomy
Hata, T., Motoi, F., Ishida, M., Naitoh, T., Katayose, Y., Egawa, S., & Unno, M. (2015) · Annals of Surgery · doi:10.1097/sla.0000000000001437 · Cited by 267 · see citing works
“10 in the Q statistic (x(2)test).(20) I(2) values ranged from 0% to 100% and were interpreted as follows: 0 to 24%, no heterogeneity; 25% to 49%, moderate heterogeneity; 50% to 74% high level of heterogeneity; 75% to 100%, extreme heterogeneity.”
BackgroundNigg, J. T., Lewis, K., Edinger, T., & Falk, M. (2012) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2011.10.015 · Cited by 260 · see citing works
Probiotics for the prevention of pediatric antibiotic-associated diarrhea
Johnston, B. C., Goldenberg, J. Z., Vandvik, P. O., Sun, X., & Guyatt, G. H. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004827.pub3 · Cited by 258 · see citing works
Wang, Q., Zhu, H., Liu, H., & Zhang, B. (2012) · Hepatology · doi:10.1002/hep.25610 · Cited by 246 · see citing works
Carter, E. C., & McCullough, M. E. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00823 · Cited by 381 · see citing works
“Data were obtained from Martin Hagger, and further uses of these data should be acknowledged as such (Hagger et al., 2010).”
MethodInfluentialDadi, A. F., Miller, E. R., & Mwanri, L. (2020) · PLoS ONE · doi:10.1371/journal.pone.0227323 · Cited by 245 · see citing works
Vassos, E., Collier, D. A., & Fazel, S. (2014) · Molecular Psychiatry · doi:10.1038/mp.2013.31 · Cited by 234 · see citing works
Knops, A. M., Legemate, D. A., Goossens, A., Bossuyt, P. M. M., & Ubbink, D. T. (2013) · Annals of Surgery · doi:10.1097/sla.0b013e3182864fd6 · Cited by 231 · see citing works
Coles, N. A., Larsen, J. T., & Lench, H. C. (2019) · Psychological Bulletin · doi:10.1037/bul0000194 · Cited by 231 · see citing works
Meta‐research: The art of getting it wrong
Ioannidis, J. P. A. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.19 · Cited by 231 · see citing works
“2010, 1 169--184 1 7 5 A number of tests have been developed by meta-analysts to detect and exclude publication bias, but I am afraid that they are used beyond their real capacity (Ioannidis, 2008a; Ioannidis and Trikalinos, 2007b; Lau et al., 2006).”
BackgroundInfluentialSelf-citationGagnier, J. J., Moher, D., Boon, H., Beyene, J., & Bombardier, C. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-111 · Cited by 222 · see citing works
Papageorgiou, S. N., Koletsi, D., Iliadi, A., Peltomäki, T., & Eliades, T. (2019) · European Journal of Orthodontics · doi:10.1093/ejo/cjz094 · Cited by 238 · see citing works
Chen, D. T., Jiang, X., Akula, N., Shugart, Y. Y., Wendland, J. R., Steele, C. J. M., Kassem, L., Park, J.-H., Chatterjee, N., Jamain, S., et al. (2011) · Molecular Psychiatry · doi:10.1038/mp.2011.157 · Cited by 208 · see citing works
Wu, B., Gu, D.-Z., Yu, J., Yang, J., & Shen, W. (2020) · Infection Genetics and Evolution · doi:10.1016/j.meegid.2020.104485 · Cited by 209 · see citing works
Koretsi, V., Zymperdikas, V. F., Papageorgiou, S. N., & Papadopoulos, M. A. (2015) · The European Journal of Orthodontics · doi:10.1093/ejo/cju071 · Cited by 191 · see citing works
“Additional analyses Possible sources of heterogeneity were sought through pre-specified mixed-effects subgroup analyses and random-effects meta-regression with the Knapp–Hartung adjustment (60).”
MethodBerlim, M. T., Neufeld, N. H., & Van den Eynde, F. (2013) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2013.03.022 · Cited by 219 · see citing works
“The lack of significant overall heterogeneity among the included RCTs and our ability to discern the likely source of heterogeneity for pre-post Y-BOCS results shows that our findings are reliable (Ioannidis, 2008).”
ResultMaxwell, L., Devries, K. M., Zionts, D. L., Alhusen, J. L., & Campbell, J. C. (2015) · PLoS ONE · doi:10.1371/journal.pone.0118234 · Cited by 188 · see citing works
“The heterogeneity of our exposure and outcome and the small number of included studies necessarily complicates the assessment of publication bias [60].”
BackgroundMethodHusebø, A. M. L., Dyrstad, S. M., Søreide, J. A., & Bru, E. (2013) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2012.04322.x · Cited by 177 · see citing works
“patients decrease their physical activity levels after diagnosis, the importance of a positive attitude towards exercise including an established exercise behaviour before the cancer diagnosis has been made, increase the adherence to exercise intervention programmes (Midtgaard et al. 2009). In a review of 38 studies i…”
BackgroundMethodInfluentialComparison of four heterogeneity measures for meta‐analysis
Lin, L. (2019) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13159 · Cited by 186 · see citing works
Moreno, S. G., Sutton, A. J., Turner, E. H., Abrams, K. R., Cooper, N. J., Palmer, T. M., & Ades, A. E. (2009) · BMJ · doi:10.1136/bmj.b2981 · Cited by 184 · see citing works
Revisiones sistemáticas de la literatura. Qué se debe saber acerca de ellas
Manterola, C., Astudillo, P., Arias, E., & Claros, N. (2011) · Cirugía Española · doi:10.1016/j.ciresp.2011.07.009 · Cited by 184 · see citing works
Interpreting Meta-Analyses of Genome-Wide Association Studies
Han, B., & Eskin, E. (2012) · PLoS Genetics · doi:10.1371/journal.pgen.1002555 · Cited by 173 · see citing works
“Another standard approach is to plot observed effect sizes and their confidence intervals of all studies [17, 25, 26].”
BackgroundAntidepressants for the treatment of depression in people with cancer.
Ostuzzi, G., Matcham, F., Dauchy, S., Barbui, C., & Hotopf, M. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011006.pub2 · Cited by 54 · see citing works
“Ioannidis 2008 Ioannidis JP. Interpretation of tests of heterogeneity and bias in meta-analysis.”
BackgroundMethodFisher, D. J., Copas, A. J., Tierney, J. F., & Parmar, M. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.11.016 · Cited by 178 · see citing works
McMahon, F. J., Akula, N., Schulze, T. G., Detera‐Wadleigh, S. D., Steele, C. J. M., Wendland, J. R., Breuer, R., Strohmaier, J., Rietschel, M., Muglia, P., et al. (2010) · Nature Genetics · doi:10.1038/ng.523 · Cited by 179 · see citing works
Autotransplantation of teeth in humans: a systematic review and meta-analysis
Almpani, K., Papageorgiou, S. N., & Papadopoulos, M. A. A. P. (2015) · Clinical Oral Investigations · doi:10.1007/s00784-015-1473-9 · Cited by 179 · see citing works
Pabalan, N., Singian, E., Tabangay, L., Jarjanazi, H., Boivin, M. J., & Ezeamama, A. E. (2018) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0005523 · Cited by 176 · see citing works
Kuramatsu, J. B., Biffi, A., Gerner, S. T., Sembill, J. A., Sprügel, M. I., Leasure, A. C., Sansing, L. H., Matouk, C. C., Falcone, G. J., Endres, M., et al. (2019) · JAMA · doi:10.1001/jama.2019.13014 · Cited by 167 · see citing works
Moore, K. A., Simpson, J. A., Scoullar, M. J. L., McGready, R., & Fowkes, F. J. I. (2017) · The Lancet Global Health · doi:10.1016/s2214-109x(17)30340-6 · Cited by 165 · see citing works
The validity of circulating microRNAs in oncology: Five years of challenges and contradictions
Jarry, J., Schadendorf, D., Greenwood, C. M. T., Spatz, A., & van Kempen, L. C. L. T. (2014) · Molecular Oncology · doi:10.1016/j.molonc.2014.02.009 · Cited by 163 · see citing works
“the results identified as “most significant” in the study are subject to winner’s curse (Ioannidis, 2008; Faye et al., 2011).”
Pereira, T., Patsopoulos, N. A., Salanti, G., & Ioannidis, J. P. A. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.13 · Cited by 161 · see citing works
Self-citationLow birth weight and its associated factors in Ethiopia: a systematic review and meta-analysis
Endalamaw, A., Engeda, E. H., Ekubagewargies, D. T., Belay, G. M., & Tefera, M. A. (2018) · The Italian Journal of Pediatrics/Italian journal of pediatrics · doi:10.1186/s13052-018-0586-6 · Cited by 161 · see citing works
“Heterogeneity of studies was quantified using the I-squared statistic, in which 25, 50, and 75% represented low, moderate and high heterogeneity respectively [35, 36].”
MethodHow effective are expressive writing interventions for adolescents? A meta-analytic review.
Travagin, G., Margola, D., & Revenson, T. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.01.003 · Cited by 155 · see citing works
Cystatin C and Cardiovascular Disease
van der Laan, S. W., Fall, T., Soumaré, A., Teumer, A., Sedaghat, S., Baumert, J. J., Zabaneh, D., van Setten, J., Išgum, I., Galesloot, T. E., et al. (2016) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2016.05.092 · Cited by 159 · see citing works
“Adding more studies to the observational analysis (46) or stratifying on the basis of these subgroups (29) might reduce heterogeneity and/or identify potential characteristics that account for heterogeneity.”
BackgroundAntidepressants for the treatment of depression in people with cancer
Ostuzzi, G., Matcham, F., Dauchy, S., Barbui, C., & Hotopf, M. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011006.pub3 · Cited by 158 · see citing works
Oral Anticoagulation and Functional Outcome after Intracerebral Hemorrhage
Biffi, A., Kuramatsu, J. B., Leasure, A. C., Kamel, H., Kourkoulis, C. E., Schwab, K. M., Ayres, A. M., Elm, J., Gurol, M. E., Greenberg, S. M., et al. (2017) · Annals of Neurology · doi:10.1002/ana.25079 · Cited by 151 · see citing works
“Mortality 105 (31) 60 (30) 55 (34) 38 (37) 30 (22) 25 (32)”
Dadi, A. F., Miller, E. R., & Mwanri, L. (2020) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-020-03092-7 · Cited by 161 · see citing works
“Visual inspection of a funnel plot asymmetry and Egger’s regression test were used to check for potential publication bias [32, 33].”
MethodLinden, A., & Hönekopp, J. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620964193 · Cited by 123 · see citing works
“(2014) reported a statistically significant moderator, but given its atheoretical nature and the number of moderators tested, it remains debatable whether this reflects progress in understanding or successful capitalization on chance (Ioannidis, 2008).”
BackgroundLeichsenring, F., Abbass, A. A., Heim, N., Keefe, J. R., Kisely, S. R., Luyten, P., Rabung, S., & Steinert, C. (2023) · World Psychiatry · doi:10.1002/wps.21104 · Cited by 147 · see citing works
Berlim, M. T., McGirr, A., den Eynde, F. V., Fleck, M. P., & Giacobbe, P. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.02.016 · Cited by 136 · see citing works
“Fifth, the reliability of the analyses of heterogeneity and publication bias might have been relatively low owing to the small number of included studies (Ioannidis, 2008).”
ResultThe link between dissociation and both suicide attempts and non-suicidal self-injury: Meta-analyses
Calati, R., Bensassi, I., & Courtet, P. (2017) · Psychiatry Research · doi:10.1016/j.psychres.2017.01.035 · Cited by 138 · see citing works
Detecting and adjusting for small‐study effects in meta‐analysis
Rücker, G., Carpenter, J. R., & Schwarzer, G. (2011) · Biometrical Journal · doi:10.1002/bimj.201000151 · Cited by 132 · see citing works
“It is questioned whether funnel plots are a suited means for judging publication bias (Tang and Liu, 2000; Terrin et al., 2003, 2005; Lau et al., 2006; Ioannidis and Trikalinos, 2007; Ioannidis, 2008).”
BackgroundInterventions for orthodontically induced white spot lesions: a systematic review and meta-analysis
Höchli, D., Hersberger-Zurfluh, M. A., Papageorgiou, S. N., & Eliades, T. (2016) · European Journal of Orthodontics · doi:10.1093/ejo/cjw065 · Cited by 136 · see citing works
“10 used for the test of heterogeneity (28).”
MethodGarcía‐Robles, P., Cortés‐Pérez, I., Nieto-Escámez, F. A., García‐López, H., Obrero‐Gaitán, E., & Osuna‐Pérez, M. C. (2024) · Anatomical Sciences Education · doi:10.1002/ase.2397 · Cited by 156 · see citing works
“49,50 The risk of publication bias was assessed through the visualiza - tion of the funnel plots, with asymmetric funnel plots indicating a risk of publication bias, 51 and the p - value of the Egger test (present if p 0.1).”
MethodInfluentialUS studies may overestimate effect sizes in softer research
Fanelli, D., & Ioannidis, J. P. A. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1302997110 · Cited by 132 · see citing works
“Thepatterns observed in this study seem tobedifferent from, and independent of, the classic file-drawer problem, which is often incorrectly equated with small-study effects (inverse correlation between study size and magnitude of reporting) (44).”
BackgroundSelf-citationZumsteg, J. M., Cooper, J. S., & Noon, M. S. (2012) · Journal of Industrial Ecology · doi:10.1111/j.1530-9290.2012.00476.x · Cited by 130 · see citing works
“Bias Across Studies Publication bias is often the main type of bias across studies; it can be a concern when published studies do not accurately reflect the actual range of studies that have been performed (Ioannidis 2008).”
Light therapy in non-seasonal depression: An update meta-analysis
Tao, L., Jiang, R., Zhang, K., Qian, Z., Chen, P., Lv, Y., & Yao, Y. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2020.113247 · Cited by 129 · see citing works
Scientists Admitting to Plagiarism: A Meta-analysis of Surveys
Pupovac, V., & Fanelli, D. (2014) · Science and Engineering Ethics · doi:10.1007/s11948-014-9600-6 · Cited by 129 · see citing works
“…be emphasized, however, that the funnel plot test is of limited use when the number of primary studies is small and between-study heterogeneity is large, because these conditions make the interpretation of funnel plots very ambiguous (Terrin et al. 2003; Macaskill et al. 2001; Ioannidis 2008).”
A meta-analysis on verbal working memory in children and adolescents with ADHD
Ramos, A. A., Hamdan, A. C., & Machado, L. (2020) · The Clinical Neuropsychologist · doi:10.1080/13854046.2019.1604998 · Cited by 125 · see citing works
“In consideration of the fact that a bias analysis-based exclusively on funnel plots is primarily visual (Egger, Smith, et al., 1997), precarious (Ioannidis, 2008) and subjective (Duval, 2005; Sterne, Becker, & Egger, 2005), its use was complemented with Duval and Tweedie’s trim-and-fill method…”
MethodGedefaw, G. A., Demis, A., Alemnew, B. T. T., Wondmieneh, A. B., Getie, ፟., & Waltengus, F. (2020) · Patient Safety in Surgery · doi:10.1186/s13037-020-00236-8 · Cited by 127 · see citing works
“Cochrane Q-test and Isquared statistics were computed to check the heterogeneity of studies [15, 16].”
MethodNaenni, N., Lim, H., Papageorgiou, S. N., & Hämmerle, C. H. F. (2019) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.13052 · Cited by 126 · see citing works
“10 used for tests of between-studies or between-subgroups heterogeneity (Ioannidis, 2008).”
MethodInfluentialKitsiou, S., Paré, G., & Jaana, M. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2770 · Cited by 118 · see citing works
“However, the limitations of these metrics [64-67,70,71] and the uncertainty around the I point estimates, which can be expressed with 95% confidence intervals, were not considered in any of the included MAs.”
BackgroundMethodInfluentialThe role of the funnel plot in detecting publication and related biases in meta-analysis
Liu, J. L. Y. (2011) · Evidence-Based Dentistry · doi:10.1038/sj.ebd.6400831 · Cited by 96 · see citing works
Bedaso, A., Mekonnen, N., & Duko, B. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058187 · Cited by 119 · see citing works
The pulpal anesthetic efficacy of articaine versus lidocaine in dentistry
Brandt, R. G., Anderson, P. F., McDonald, N. J., Sohn, W., & Peters, M. C. (2011) · The Journal of the American Dental Association · doi:10.14219/jada.archive.2011.0219 · Cited by 119 · see citing works
Albaladejo-Saura, M., Vaquero‐Cristóbal, R., González-Gálvez, N., & Esparza-Ros, F. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18010328 · Cited by 120 · see citing works
“The DerSimonianLaird (Cohen) pooling method was used, and heterogeneity was assessed using the Cochrane Q test (Chi2), Higgins I2, and significance (p) to determine the appropriateness of the application of a fixed or random-effect model for the pooled analysis [50].”
MethodCoronado-Montoya, S., Levis, A. W., Kwakkenbos, L., Steele, R. J., Turner, E. H., & Thombs, B. D. (2016) · PLOS ONE · doi:10.1371/journal.pone.0153220 · Cited by 107 · see citing works
“Generally, risk of reporting bias will be higher in fields with small, underpowered trials [68]; when there is a strong incentive for reporting positive results, which is often the case when professionals who practice a given psychological treatment also test it [69, 70]; and when there is prior documentation of bias…”
BackgroundApplication of MALDI-TOF MS to rapid identification of anaerobic bacteria
Li, Y., Shan, M., Zhu, Z., Mao, X., Yan, M., Chen, Y., Zhu, Q., Li, H., & Gu, B. (2019) · BMC Infectious Diseases · doi:10.1186/s12879-019-4584-0 · Cited by 109 · see citing works
“This result is called “publication bias” in statistics [17].”
BackgroundBasic study design influences the results of orthodontic clinical investigations
Papageorgiou, S. N., Xavier, G. M., & Cobourne, M. T. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.03.008 · Cited by 108 · see citing works
“10 used for the test of heterogeneity and reporting biases [30].”
MethodSaavedra, J. S. M., Vergara-Méndez, L. D., Pradilla, I. F., Vélez-van-Meerbeke, A., & Talero‐Gutiérrez, C. (2018) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2018.07.008 · Cited by 108 · see citing works
(ICCBD), M. O. T. I. C. C. F. B. D., Song, J., Bergen, S. E., Florio, A. D., Karlsson, R., Charney, A. W., Ruderfer, D. M., Stahl, E. A., Chambert, K. D., Moran, J. L., et al. (2015) · Molecular Psychiatry · doi:10.1038/mp.2015.165 · Cited by 108 · see citing works
Ardeshirlarijani, E., Tabatabaei‐Malazy, O., Mohseni, S., Qorbani, M., Larijani, B. A., & Jalili, R. B. (2019) · DARU Journal of Pharmaceutical Sciences · doi:10.1007/s40199-019-00302-2 · Cited by 108 · see citing works
Empirically Supported Treatment: Recommendations for a New Model
Tolin, D. F., McKay, D., Forman, E. M., Klonsky, E. D., & Thombs, B. D. (2015) · Clinical Psychology: Science and Practice · doi:10.1111/cpsp.12122 · Cited by 219 · see citing works
Rieker, J. A., Avilés, J. M. R., Muiños, M., & Ballesteros, S. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.838968 · Cited by 108 · see citing works
“Nonetheless, it has been suggested that large estimates of between-study heterogeneity can cause regression asymmetry (Ioannidis and Trikalinos, 2007; Ioannidis, 2008).”
BackgroundEvidence‐based healthcare, clinical knowledge and the rise of personalised medicine
Miles, A., Loughlin, M., & Polychronis, A. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.01094.x · Cited by 99 · see citing works
“The ultimate paper which closes the 2008 Thematic Edition on EBM has been contributed by John Ioannidis and is concerned with the interpretation of tests of heterogeneity and bias in metaanalysis [273].”
BackgroundKonstantonis, D., Vasileiou, D., Papageorgiou, S. N., & Eliades, T. (2018) · European Journal Of Oral Sciences · doi:10.1111/eos.12409 · Cited by 100 · see citing works
“10 that was used for the test of heterogeneity and reporting biases because of low power (33).”
MethodEffects of exercise programs on kyphosis and lordosis angle: A systematic review and meta-analysis
González-Gálvez, N., Gea-García, G. M., & Marcos-Pardo, P. J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0216180 · Cited by 98 · see citing works
Mitsikostas, D., Chalarakis, N., Mantonakis, L., Delicha, E. M., & Sfikakis, P. P. (2011) · European Journal of Neurology · doi:10.1111/j.1468-1331.2011.03528.x · Cited by 97 · see citing works
Wang, S., Zhang, L., Pan, W., & Hua, H. (2017) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-017-1543-z · Cited by 94 · see citing works
“25, the effect size heterogeneity was considered statistically significant [45].”
BackgroundTsilidis, K. K., Papatheodorou, S., Εvangelou, Ε., & Ioannidis, J. P. A. (2012) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djs437 · Cited by 93 · see citing works
“Heterogeneity may often be a manifestation of bias in some studies of a metaanalysis, and this may be difficult to dissect and differentiate from genuine differences across studies (65).”
BackgroundSelf-citationDepression among caregivers of cancer patients: Updated systematic review and meta‐analysis
Bedaso, A., Kibret, G. D., & Duko, B. (2022) · Psycho-Oncology · doi:10.1002/pon.6045 · Cited by 92 · see citing works
“We have examined publication bias by visual inspection of a funnel plot and Egger's regression test.(29,30) A p‐value <0.”
MethodWang, C., Yu, J., Wang, H.-F., Tan, C., Meng, X., & Tan, L. (2014) · Journal of Alzheimer s Disease · doi:10.3233/jad-140660 · Cited by 91 · see citing works
“Considering the relatively small number of methodologically sound studies, it is not yet mature enough for the investigation of publication bias using funnel plots [13].”
BackgroundCutts, J. C., Agius, P. A., Lin, Z., Powell, R., Moore, K. A., Draper, B. L., Simpson, J. A., & Fowkes, F. J. I. (2020) · BMC Medicine · doi:10.1186/s12916-019-1467-6 · Cited by 90 · see citing works
“If the I(2) statistic was ≤ 75%, a metaanalysis based on a random effects model was conducted; when the I(2) statistic was > 75% and/or the lower 95% confidence limit was between 50 and 100%, the studies were not combined [49, 50].”
MethodPeripheral vascular endothelial growth factor as a novel depression biomarker: A meta-analysis
Carvalho, A. F., Köhler, C. A., McIntyre, R. S., Knöchel, C., Brunoni, A. R., Thase, M. E., Quevedo, J., Fernandes, B. S., & Berk, M. (2015) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2015.07.002 · Cited by 86 · see citing works
“The I2 measure was used to quantify between-study heterogeneity, and considered to be high when I2 > 50% (Ioannidis, 2008).”
MethodMcFarland, L. V., & Goh, S. (2019) · Travel Medicine and Infectious Disease · doi:10.1016/j.tmaid.2018.09.007 · Cited by 86 · see citing works
Azevedo, L. B., Stephenson, J. R., Ells, L. J., Adu‐Ntiamoah, S., DeSmet, A., Giles, E. L., Haste, A., O’Malley, C. L., Jones, D., Chai, L. K., et al. (2021) · Obesity Reviews · doi:10.1111/obr.13373 · Cited by 87 · see citing works
Nocebo in clinical trials for depression: A meta-analysis
Mitsikostas, D. D., Mantonakis, L., & Chalarakis, N. (2013) · Psychiatry Research · doi:10.1016/j.psychres.2013.10.019 · Cited by 86 · see citing works
“Data was analyzed using a random effects model, due to heterogeneity between studies, which was assessed by I2 and Cochran Q tests (Cochran, 1954; Higgins and Thompson, 2002; Ioannidis, 2008).”
Papageorgiou, S. N., Gölz, L., Jäger, A., Eliades, T., & Bourauel, C. P. (2016) · European Journal Of Oral Sciences · doi:10.1111/eos.12250 · Cited by 86 · see citing works
Berlim, M. T., & Van den Eynde, F. (2014) · The Canadian Journal of Psychiatry · doi:10.1177/070674371405900905 · Cited by 96 · see citing works
Exposure to pesticides and childhood leukemia risk: A systematic review and meta-analysis
Karalexi, M. A., Tagkas, C. F., Markozannes, G., Tseretopoulou, X., Hernández, A. F., Schüz, J., Halldorsson, T. I., Ψαλτοπούλου, Θ., Petridou, E. T., Tzoulaki, I., et al. (2021) · Environmental Pollution · doi:10.1016/j.envpol.2021.117376 · Cited by 85 · see citing works
Who benefits from individual placement and support? A meta-analysis
de Winter, L., Couwenbergh, C., van Weeghel, J., Sanches, S. A., Michon, H. W. C., & Bond, G. R. (2022) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796022000300 · Cited by 88 · see citing works
Buck, L. M., Dalci, O., Darendeliler, M. A., Papageorgiou, S. N., & Papadopoulou, A. K. (2016) · European Journal of Orthodontics · doi:10.1093/ejo/cjw048 · Cited by 84 · see citing works
“Statistical significance was set at a two-sided α of 5 per cent, except for test of heterogeneity, where α was set at 10 per cent, due to low power (48).”
BackgroundEfficacy and safety of miconazole for oral candidiasis: a systematic review and meta‐analysis
Zhang, L., Fu, J., Hua, H., & Yan, Z. (2015) · Oral Diseases · doi:10.1111/odi.12380 · Cited by 84 · see citing works
- Authors
- Ioannidis, J. P. A.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- Journal of Evaluation in Clinical Practice
- Publisher
- Wiley
- Published
- 2008-10
- Volume
- 14
- Issue
- 5
- Pages
- 951-957
- ISSN
- 1356-1294 (ISSN-L)
- 1365-2753
- OpenAlex
- W2144112749
- Cited by
- 696
- References
- 57
- Citations by year
- 2026: 52 · 2025: 69 · 2024: 98 · 2023: 72 · 2022: 50 · 2021: 64 · 2020: 49 · 2019: 34 · 2018: 30 · 2017: 23 · 2016: 23 · 2015: 26 · 2014: 27 · 2013: 22 · 2012: 15
- FWCI
- 4.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Evaluation in Clinical Practice (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- heterogeneity testing, publication bias, meta-analysis, statistical heterogeneity, clinical heterogeneity, false positive signals, bias detection
- MeSH terms
- Data Interpretation, Statistical, Humans, Regression Analysis, Research Design, Risk, Sensitivity and Specificity, Meta-Analysis as Topic, Reproducibility of Results, Bias, Confidence Intervals, Publication Bias, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
