The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Interpretation of tests of heterogeneity and bias in meta-analysis
Abstract
Statistical tests of heterogeneity and bias, in particular publication bias, are very popular in meta-analyses. These tests use statistical approaches whose limitations are often not recognized. Moreover, it is often implied with inappropriate confidence that these tests can provide reliable answers to questions that in essence are not of statistical nature. Statistical heterogeneity is only a correlate of clinical and pragmatic heterogeneity and the correlation may sometimes be weak. Similarly, statistical signals may hint to bias, but seen in isolation they cannot fully prove or disprove bias in general, let alone specific causes of bias, such as publication bias in particular. Both false-positive and false-negative signals of heterogeneity and bias can be common and their prevalence may be anticipated based on some rational considerations. Here I discuss the major common challenges and flaws that emerge in using and interpreting statistical tests of heterogeneity and bias in meta-analyses. I discuss misinterpretations that can occur at the level of statistical inference, clinical/pragmatic inference and specific cause attribution. Suggestions are made on how to avoid these flaws, use these tests properly and learn from them.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These limitations are common in meta-analyses of cross-sectional studies (Ioannidis, 2008).”
From Decision making capacity for treatment in psychiatric inpatients: a systematic review and meta-analysis (2024) · cites this paper as Ioannidis, 2008
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationThe combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Self-citationAssessing heterogeneity in meta-analysis: Q statistic or I² index?
Huedo-Medina, T. B., Sánchez-Meca, J., Marín-Martínez, F., & Botella, J. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.2.193 · Cited by 4,256
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
Cited by
Sorted by citations
Items 101–200 of 695
Papageorgiou, S. N., Reichert, C., Jäger, A., & Deschner, J. (2015) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.12365 · Cited by 84
“All p values were two-sided with a set at 5%, except for the test of heterogeneity, where a was set at 10% (Ioannidis 2008).”
MethodChen, W., Jiang, H., Cai, B., & Klassen, K. (2010) · Nutrition · doi:10.1016/j.nut.2009.09.011 · Cited by 82
Gränicher, P., Mulder, L. T. M. A., Lenssen, A. F., Scherr, J., Swanenburg, J., & de Bie, R. A. (2022) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2022.11160 · Cited by 85
Coyne, J. C., Thombs, B. D., & Hagedoorn, M. (2010) · Health Psychology · doi:10.1037/a0017633 · Cited by 84
“Even in the case of a true null, such a collection of published small studies would be expected to produce a robust effect size estimate (Howard et al., 2009; Ioannidis, 2008a; Kraemer et al., 1998).”
BackgroundResultBias from historical control groups used in orthodontic research: a meta-epidemiological study
Papageorgiou, S. N., Koretsi, V., & Jäger, A. (2016) · European Journal of Orthodontics · doi:10.1093/ejo/cjw035 · Cited by 80
“10 used for the test of heterogeneity and reporting biases (20).”
MethodPapageorgiou, S. N., Höchli, D., & Eliades, T. (2017) · The Korean Journal of Orthodontics · doi:10.4041/kjod.2017.47.6.401 · Cited by 77
Cutts, J. C., Powell, R., Agius, P. A., Beeson, J. G., Simpson, J. A., & Fowkes, F. J. I. (2014) · BMC Medicine · doi:10.1186/s12916-014-0150-1 · Cited by 78
“1, a pooled effect was not estimated [18-21].”
Stroke case fatality in sub-Saharan Africa: Systematic review and meta-analysis
Adoukonou, T., Kossi, O., Fotso, P. M., Agbétou, M., Magné, J., Gbaguidi, G. N., Houinato, D. S., Preux, P., & Lacroix, P. (2021) · International Journal of Stroke · doi:10.1177/1747493021990945 · Cited by 76
“The I(2) value >50% indicated heterogeneity.(20) We used STATA software, StataCorp, TX, USA(21) for the quantitative analysis.”
MethodPrevalence of perinatal asphyxia in East and Central Africa: systematic review and meta-analysis
Workineh, Y., Kasa, A. S., Ayalew, E., Animaw, W., Bayih, M. T., & Birhanu, M. (2020) · Heliyon · doi:10.1016/j.heliyon.2020.e03793 · Cited by 75
“Heterogeneity of studies was quantified using the I-Squared Statistic, in which 25, 50, and 75% represented low, moderate and high heterogeneity respectively [28, 29].”
MethodMethicillin resistant Staphylococcus aureus in Ethiopia: a meta-analysis
Eshetie, S., Kumie, F. T., Moges, F., Amsalu, A., Birhan, W., & Huruy, K. (2016) · BMC Infectious Diseases · doi:10.1186/s12879-016-2014-0 · Cited by 75
“The small study bias was measured by Begg’s funnel plot [14].”
MethodSystematic review and meta-analysis of glyphosate exposure and risk of lymphohematopoietic cancers
Chang, E. T., & Delzell, E. (2016) · Journal of Environmental Science and Health Part B · doi:10.1080/03601234.2016.1142748 · Cited by 74
Statin Treatment and Functional Outcome After Ischemic Stroke
Biffi, A., Devan, W. J., Anderson, C. D., Cortellini, L., Furie, K. L., Rosand, J. M., & Rost, N. S. (2011) · Stroke · doi:10.1161/strokeaha.110.605923 · Cited by 74
Statin use and outcome after intracerebral hemorrhage
Biffi, A., Devan, W. J., Anderson, C. D., Ayres, A. M., Schwab, K. M., Cortellini, L., Viswanathan, A., Rost, N. S., Smith, E. E., Goldstein, J. N., et al. (2011) · Neurology · doi:10.1212/wnl.0b013e3182194be9 · Cited by 74
Antidepressants for the treatment of depression in people with cancer
Vita, G., Compri, B., Matcham, F., Barbui, C., & Ostuzzi, G. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011006.pub4 · Cited by 73
Chi, N., Huang, Y.-C., Chiu, H.-Y., Chang, H., & Huang, H. (2019) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2019.10.181 · Cited by 72
Sanabria, Á., Kowalski, L. P., Bradley, P. J., Hartl, D. M., Bradford, C. R., de Bree, R., Rinaldo, A., & Ferlito, A. (2011) · Head & Neck · doi:10.1002/hed.21722 · Cited by 72
“In our opinion, the clinical and statistical heterogeneity of these studies makes it impossible to combine their results in a pooled one, precluding the use of a meta-analysis to evaluate the SCM flap as a measure to avoid Frey’s syndrome.(45) Most heterogeneity among the studies can be explained by clinical and metho…”
MethodParoxetine versus other anti-depressive agents for depression
Purgato, M., Papola, D., Gastaldon, C., Trespidi, C., Magni, L. R., Rizzo, C., Furukawa, T. A., Watanabe, N., Cipriani, A., & Barbui, C. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006531.pub2 · Cited by 73
Papageorgiou, S. N., Konstantinidis, I., Papadopoulou, K., Jäger, A., & Bourauel, C. P. (2013) · European Journal of Orthodontics · doi:10.1093/ejo/cjt064 · Cited by 72
Berlim, M. T., Broadbent, H. J., & den Eynde, F. V. (2013) · The International Journal of Neuropsychopharmacology · doi:10.1017/s1461145712001691 · Cited by 72
“However, it is difficult to determine the precise nature of this heterogeneity as it may be due, for example, to genuine differences in participants, interventions, co-interventions, outcomes, measurements, settings and to numerous other factors varying across data sets, studies and participants (Thompson, 1994; Petit…”
BackgroundPrevention of orthodontic enamel demineralization: A systematic review with meta‐analyses
Tasios, T., Papageorgiou, S. N., Papadopoulos, M. A. A. P., Τσάπας, Α., & Haidich, A. (2019) · Orthodontics and Craniofacial Research · doi:10.1111/ocr.12322 · Cited by 74
Abbing, A., Koretsi, V., Eliades, T., & Papageorgiou, S. N. (2020) · Progress in Orthodontics · doi:10.1186/s40510-020-00334-4 · Cited by 79
“All P values were two-sided with α = 5%, except for the test of between-studies or between-subgroups heterogeneity, where α value was set at 10% [28].”
MethodAbate, B. B., Bimerew, M., Kassie, A. M., Kassaw, M. W., & Azmeraw, M. (2020) · BMC Pediatrics · doi:10.1186/s12887-020-02083-z · Cited by 67
“In this study, the I(2) statistic value of zero indicates true homogeneity, whereas the value 25, 50, and 75% represented low, moderate and high heterogeneity respectively [18, 19].”
BackgroundImproved Overall Survival of Colorectal Cancer under Multidisciplinary Team: A Meta‐Analysis
Peng, D., Cheng, Y.-X., & Cheng, Y. (2021) · BioMed Research International · doi:10.1155/2021/5541613 · Cited by 66
“The value of I2 and the result of the Chi-squared test were used to assess the statistical heterogeneity [25, 26].”
MethodEpidemiology of depression among displaced people: A systematic review and meta-analysis
Bedaso, A., & Duko, B. (2022) · Psychiatry Research · doi:10.1016/j.psychres.2022.114493 · Cited by 67
Chronic endometritis and recurrent reproductive failure: a systematic review and meta-analysis
Ticconi, C., Inversetti, A., Marraffa, S., Campagnolo, L., Arthur, J., Zambella, E., & Simone, N. D. (2024) · Frontiers in Immunology · doi:10.3389/fimmu.2024.1427454 · Cited by 69
“These criteria were set according to Higgins et al. (14) and Ioannidis (15).”
BackgroundZhao, X., Ren, Y., Hu, Y., Cui, N., Wang, X., & Cui, Y. (2018) · PLoS ONE · doi:10.1371/journal.pone.0202185 · Cited by 66
Hussen, S., & Tadesse, B. T. (2019) · BioMed Research International · doi:10.1155/2019/4562385 · Cited by 63
“05 suggests presence of a statistically significant publication bias [19].”
BackgroundStathis, P. G., Smpiliris, M. E., Konitsiotis, S., & Mitsikostas, D. D. (2012) · European Journal of Neurology · doi:10.1111/ene.12014 · Cited by 65
Papageorgiou, S. N., Antonoglou, G. N., Michelogiannakis, D., Kakali, L., Eliades, T., & Madianos, P. (2022) · Journal of Clinical Periodontology · doi:10.1111/jcpe.13529 · Cited by 75
Jorgensen, J. E., Waljee, A. K., Volk, M. L., Sonnenday, C. J., Elta, G. H., Al-Hawary, M. M., Singal, A. G., Taylor, J. R., & Elmunzer, B. J. (2011) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2010.12.014 · Cited by 64
Belayneh, Z., Chavulak, J., Lee, D. A., Petrakis, M., & Haines, T. P. (2024) · Journal of Clinical Nursing · doi:10.1111/jocn.17041 · Cited by 70
Cassina, C., Papageorgiou, S. N., & Eliades, T. (2017) · European Journal of Orthodontics · doi:10.1093/ejo/cjx047 · Cited by 63
Papageorgiou, S. N., Eliades, T., & Hämmerle, C. H. F. (2018) · Clinical Oral Implants Research · doi:10.1111/clr.13291 · Cited by 62
“between-studies or between-subgroups heterogeneity (Ioannidis 2008).”
MethodInfluentialLi, Y., Yang, C., Zhang, W., Li, S., Li, Q., Chen, L., Lui, S., Kemp, G. J., Biswal, B. B., Shah, N. J., et al. (2021) · Frontiers in Neuroendocrinology · doi:10.1016/j.yfrne.2021.100915 · Cited by 61
“When I2 < 50% or P > 0.10 signaled no significant between-study heterogeneity (Ioannidis, 2008), the pooled estimate of SMD would be obtained using the fixed-effects model; otherwise, if heterogeneity was significant, a random-effects model would be used (Higgins and Thompson, 2002).”
MethodInfluentialAbate, B. B., Sendekie, A. K., Tadesse, A. W., Engdaw, T., Mengesha, A., Zemariam, A. B., Alamaw, A. W., Abebe, G., & Azmeraw, M. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1391312 · Cited by 60
“I 2 of 25%, 50%, and 75% thresholds for low, moderate, and high heterogeneity, respectively, with a threshold of I² 50%, is often considered indicative of low heterogeneity (25, 26).”
BackgroundAbate, B. B., Sendekie, A. K., Faris, A. M., Abebe, G. K., Azmeraw, M., Alamaw, A. W., Zemariam, A. B., Kitaw, T. A., Kassaw, A., Wodaynew, T., et al. (2024) · Neuropsychobiology · doi:10.1159/000542392 · Cited by 66
“…[44] Ayano G et al 2022 [47] McKay MT et al 2020 [48] Agnew-Blais J et al 2016 [53] Subtotal (I-squared = 81.2%, p = 0.021) Norman RE et al 2012 [54] Physical abuse Subtotal (I-squared = 95.4%, p = 0.000) Gallo EAG et al 2018 [41] Gallo EAG et al 2018 [41] Chen LP et al 2010 [55] Lindert J et…”
InfluentialClinical outcomes of HIV ‐exposed, HIV ‐uninfected children in sub‐Saharan Africa
le Roux, S. M., Abrams, E. J., Nguyen, K. K., & Myer, L. (2016) · Tropical Medicine & International Health · doi:10.1111/tmi.12716 · Cited by 59
Papageorgiou, S. N., Dimitraki, D., Kotsanos, N., Bekes, K., & van Waes, H. (2017) · Journal of Dentistry · doi:10.1016/j.jdent.2017.08.004 · Cited by 59
“10 used for tests of between-studies or between-subgroups heterogeneity [37].”
MethodBusenhart, D. M., Erb, J., Rigakos, G., Eliades, T., & Papageorgiou, S. N. (2018) · Oral Oncology · doi:10.1016/j.oraloncology.2018.06.001 · Cited by 62
Effect of orthodontic treatment on the subgingival microbiota: A systematic review and meta‐analysis
Papageorgiou, S. N., Xavier, G. M., Cobourne, M. T., & Eliades, T. (2018) · Orthodontics & Craniofacial Research · doi:10.1111/ocr.12237 · Cited by 56
“4 | Data synthesis As the effect of orthodontic therapy on subgingival microbiota is bound to be affected by many factors, including type of orthodontic appliances, treatment duration, patient oral hygiene, and biological profile,(23-25) a random-effects model was deemed appropriate to incorporate this variability.(16…”
MethodMoreno, S. G., Sutton, A. J., Ades, A. E., Cooper, N. J., & Abrams, K. R. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.009 · Cited by 54
“Although traditionally publication biases have been seen as the main reason for the observed smallstudy effects in most meta-analyses [9,31,32], the smallstudy effects phenomenon can be induced by a variety of factors besides publication biases [23,33,34].”
BackgroundThe Association of Thyroid Disease and Oral Lichen Planus: A Literature Review and Meta-analysis
Li, D. X., Li, J., Li, C., Chen, Q., & Hua, H. (2017) · Frontiers in Endocrinology · doi:10.3389/fendo.2017.00310 · Cited by 55
“The I2 test was performed to evaluate the heterogeneity among the studies, with possible values of 0–100%; I2 values of 25 and 50% were used as cutoffs for modest and high heterogeneity (22).”
MethodHussen, S., Wachamo, D., Kassa, Z. Y., & Tadesse, E. (2018) · BMC Infectious Diseases · doi:10.1186/s12879-018-3477-y · Cited by 55
“05 is statistically significance, and there is publication bias [17].”
BackgroundMusic is an effective intervention for the management of pain: An umbrella review.
Martín-Saavedra, J. S., Vergara-Méndez, L. D., & Talero-Gutiérrez, C. (2018) · Complementary Therapies in Clinical Practice · doi:10.1016/j.ctcp.2018.06.003 · Cited by 34
Is bipolar disorder associated with accelerating aging? A meta-analysis of telomere length studies
Colpo, G. D., Leffa, D. D., Köhler, C. A., Kapczinski, F. P., Quevedo, J., & Carvalho, A. F. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.06.034 · Cited by 53
Papageorgiou, S. N., Antonoglou, G. N., Tsiranidou, E., Jepsen, S. V., & Jäger, A. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.04.002 · Cited by 53
“10 used for the heterogeneity and small-study effect tests [34].”
MethodLeonhard, M. J., Chang, E. T., Loccisano, A. E., & Garry, M. R. (2019) · Toxicology · doi:10.1016/j.tox.2019.03.004 · Cited by 52
Momeni, H., & Ebrahimkhanlou, A. (2022) · Smart Materials and Structures · doi:10.1088/1361-665x/ac50f4 · Cited by 53
“Such challenges include highdimensionality, big data [31], heterogeneity of data [32], insufficient available data are four categories shown in figure 1.”
Bills, E., Greene, D., Stackpole, R., & Egan, S. J. (2023) · Appetite · doi:10.1016/j.appet.2023.106586 · Cited by 69
Almuwallad, A., Cole, E. M., Ross, J., Perkins, Z., & Davenport, R. A. (2021) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000003120 · Cited by 52
“of the four studies were observational cohort studies which are known to be at risk of confounding and bias because of lack of randomization.(11,12,25) We attempted to account for these risks, plus the heterogeneity inherent to observational studies, by using a random effects model in the meta-analysis.”
BackgroundGong, C., Wang, J., Bai, Z., Rich, D. Q., & Zhang, Y. (2021) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.150744 · Cited by 52
Adverse fetal outcomes and its associated factors in Ethiopia: a systematic review and meta-analysis
Gedefaw, G. A., Alemnew, B. T. T., & Demis, A. (2020) · BMC Pediatrics · doi:10.1186/s12887-020-02176-9 · Cited by 51
“The heterogeneity of studies was computed using the Cochrane Q-test and I-squared statistic [18, 19].”
MethodPhysiotherapy Intervention in Alzheimer's Disease: Systematic Review and Meta-Analysis
Zhu, X., Yu, Y., Wang, H.-F., Jiang, T., Cao, L., Wang, C., Wang, J., Tan, C., Meng, X., Tan, L., et al. (2015) · Journal of Alzheimer s Disease · doi:10.3233/jad-141377 · Cited by 51
“When the number of methodologically sound studies is relative small (less than 10), it is not yet mature enough to use funnel plots to investigate publication bias [21].”
Zhang, L. ‐., Zhou, P., Wei, P., Cong, X., Wu, L., & Hua, H. (2018) · Scandinavian Journal of Immunology · doi:10.1111/sji.12649 · Cited by 51
“In addition, I(2) was performed to evaluate the heterogeneity of the studies, which ranged from 0% to 100%; 25% and 50% were used as cut-offs for modest and high heterogeneity, respectively.(28) If no significant heterogeneity was detected, a fixed-effect model would be used to calculate the pooled effect size.”
MethodThe burden of healthcare-associated infection in Ethiopia: a systematic review and meta-analysis
Alemu, A. Y., Endalamaw, A., & Bayih, W. A. (2020) · Tropical Medicine and Health · doi:10.1186/s41182-020-00263-2 · Cited by 51
“Heterogeneity of studies was quantified using the I-squared statistic, in which 25, 50, and 75% represented low, moderate, and high heterogeneity, respectively [34].”
MethodMateo‐Orcajada, A., González-Gálvez, N., Abenza‐Cano, L., & Vaquero‐Cristóbal, R. (2022) · Journal of Youth and Adolescence · doi:10.1007/s10964-021-01552-7 · Cited by 51
“The Dersimonian–Laird (Cohen) pooling method was used, and heterogeneity was assessed using the Cochrane Q test (chi2), Higgins I2, and significance (p) to determine the appropriateness of the application of a fixed or random-effect model for the pooled analysis (Ioannidis 2008).”
MethodHussen, S., Mulatu, G., & Kassa, Z. Y. (2019) · Annals of Clinical Microbiology and Antimicrobials · doi:10.1186/s12941-019-0321-1 · Cited by 50
Lima, J. P. D. S. N., dos Santos, L. V., Sasse, E. C., Lima, C. S. P., & Sasse, A. D. (2010) · Journal of Thoracic Oncology · doi:10.1097/jto.0b013e3181f2451c · Cited by 50
Hypertension and intracerebral hemorrhage recurrence among white, black, and Hispanic individuals
Rodriguez-Torres, A., Murphy, M. P., Kourkoulis, C. E., Schwab, K. M., Ayres, A. M., Moomaw, C. J., Kwon, S. Y., Berthaud, J. V., Gurol, M. E., Greenberg, S. M., et al. (2018) · Neurology · doi:10.1212/wnl.0000000000005729 · Cited by 49
Biffi, A., Anderson, C. D., Nalls, M. A., Rahman, R. M. A., Sonni, A., Cortellini, L., Rost, N. S., Matarín, M., Hernandez, D. G., Plourde, A., et al. (2010) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2010.05.005 · Cited by 49
“20.(27) Effectiveness of PS control was assessed by computing mitochondrial genomic inflation factors (mtGIFs) for all analyses.”
MethodNurses' intention to leave their job in sub-Saharan Africa: A systematic review and meta-analysis
Ayalew, E., Workineh, Y., Kasa, A. S., Abate, T. W., Kerie, S., Gedamu, H., & Zeleke, B. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e07382 · Cited by 49
The role of tigecycline in the treatment of infections in light of the new black box warning
Dixit, D., Madduri, R. P., & Sharma, R. (2014) · Expert Review of Anti-infective Therapy · doi:10.1586/14787210.2014.894882 · Cited by 49
“Despite the results in a meta-analysis, statistical heterogeneity may be indeterminate and cannot support definitive conclusions [14].”
ResultEffectiveness of early orthopaedic treatment with headgear: a systematic review and meta-analysis
Papageorgiou, S. N., Kutschera, E., Beisel-Memmert, S., Gölz, L., Jäger, A., Bourauel, C. P., & Eliades, T. (2016) · European Journal of Orthodontics · doi:10.1093/ejo/cjw041 · Cited by 49
“10 used for the test of heterogeneity and reporting biases, due to low power (24).”
MethodBahar, B. S. K. B., Alkhalidy, S. R., Kaklamanos, E. G., & Athanasiou, A. E. (2019) · International Orthodontics · doi:10.1016/j.ortho.2019.08.025 · Cited by 48
Changes in social functioning over the course of psychotic disorders–A meta-analysis
de Winter, L., Couwenbergh, C., van Weeghel, J., Hasson-Ohayon, I., Vermeulen, J. M., Mulder, C. L., Boonstra, N., Klaver, K. M., Oud, M., de Haan, L., et al. (2022) · Schizophrenia Research · doi:10.1016/j.schres.2021.11.010 · Cited by 46
Papageorgiou, S. N., Papadelli, A. A., & Eliades, T. (2018) · European Journal of Orthodontics · doi:10.1093/ejo/cjx052 · Cited by 57
Hosseini, H. R., Kaklamanos, E. G., & Athanasiou, A. E. (2017) · PLoS ONE · doi:10.1371/journal.pone.0181768 · Cited by 48
Wang, C., Zhang, C., Li, S., Yu, L., Tian, F., Zhao, J., Zhang, H., Chen, W., & Zhai, Q. (2020) · Foods · doi:10.3390/foods9111540 · Cited by 48
“The I2 value of heterogeneity (I2) was classified as per previous studies [18,26]: <25% (represents low heterogeneity), 25–75% (represents moderate heterogeneity), and >75% (represents high heterogeneity).”
MethodSystematic Reviews of the Literature: What Should be Known About Them
Manterola, C., Astudillo, P., Arias, E., & Claros, N. (2013) · Cirugía Española (English Edition) · doi:10.1016/j.cireng.2013.07.003 · Cited by 46
“28 –30 Until now, we have only given a brief explanation of the MA format for studying CT with or without random assignation and eventually cohort studies.”
Piacentini, S., Polimanti, R. A., Simonelli, I., Donno, S., Pasqualetti, P., Manfellotto, D., & Fuciarelli, M. (2013) · Molecular Biology Reports · doi:10.1007/s11033-012-2405-2 · Cited by 46
Bedaso, A., Ayalew, M., Mekonnen, N., & Duko, B. (2020) · Depression Research and Treatment · doi:10.1155/2020/3695209 · Cited by 45
“Possible publication bias was assessed by inspection of the funnel plot and Egger ’ s regression tests [22, 23].”
MethodMakhmari, S. A. A., Kaklamanos, E. G., & Athanasiou, A. E. (2017) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/j.ajodo.2017.09.003 · Cited by 44
Alccayhuaman, K. A. A., Soto‐Peñaloza, D., Nakajima, Y., Papageorgiou, S. N., Botticelli, D., & Lang, N. P. (2018) · Clinical Oral Implants Research · doi:10.1111/clr.13279 · Cited by 44
“A two‐sided p ≤ 0.05 was considered significant for hypothesis testing, except for p ≤ 0.10 used for tests of between‐studies or between‐subgroups heterogeneity (Ioannidis, 2008).”
MethodInfluentialLussier, M. E., Evans, H. J., Wright, E. A., & Gionfriddo, M. R. (2019) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2019.07.002 · Cited by 45
Alemu, A. Y., Endalamaw, A., Belay, D. M., Mekonen, D. K., Birhane, B. M., & Bayih, W. A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0241073 · Cited by 43
“Heterogeneity of studies was quantified using the I-squared statistic, in which 25%, 50%, and 75% represented low, moderate, and high heterogeneity, respectively [33].”
MethodThe effect of metformin on cognitive function: A systematic review and meta-analysis
Tabatabaei‐Malazy, O., Bandarian, F., Qorbani, M., Mohseni, S., Mirsadeghi, S., Peimani, M., & Larijani, B. A. (2022) · Journal of Psychopharmacology · doi:10.1177/02698811211057304 · Cited by 43
“The degree of heterogeneity was estimated using I2 statistics (Ioannidis, 2008).”
MethodOrtiz, M., Meroño, L., Morales-Belando, M. T., Vaquero‐Cristóbal, R., & González-Gálvez, N. (2023) · Children · doi:10.3390/children10030573 · Cited by 43
“effect model for the pooled analysis [46].”
How identity fusion predicts extreme pro-group orientations: A meta-analysis
Varmann, A. H., Kruse, L., Bierwiaczonek, K., Gómez, Á., Vázquez, A., & Kunst, J. R. (2023) · European Review of Social Psychology · doi:10.1080/10463283.2023.2190267 · Cited by 54
“Generally, it is important to note that, although many moderators were significant, they could not meaningfully reduce the observed heterogeneity of effects that was very high according to common standards (Ioannidis, 2008).”
BackgroundSubstance use among young people in sub-Saharan Africa: a systematic review and meta-analysis
Ebrahim, J., Adams, J., & Demant, D. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1328318 · Cited by 65
“A random effect model, which accounts for potential heterogeneity (89), was used to estimate the pooled effects in the (90, 91).”
MethodYoung, I., Waddell, L. A., Sánchez, J., Wilhelm, B., McEwen, S. A., & Rajić, A. (2013) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2013.11.009 · Cited by 48
Sanabria, Á., Rojas, A. P., & Arevalo, J. (2019) · British journal of surgery · doi:10.1002/bjs.11216 · Cited by 43
Brini, S., Brudasca, N. I., Hodkinson, A., Kaluzinska, K., Wach, A., Storman, D., Prokop-Dorner, A., Jemioło, P., & Bala, M. M. (2023) · Clinical Psychology Review · doi:10.1016/j.cpr.2022.102236 · Cited by 44
Saulle, R., Semyonov, L., Mannocci, A., Careri, A., Saburri, F., Ottolenghi, L., Guerra, F., & Torre, G. L. (2014) · Oral Diseases · doi:10.1111/odi.12269 · Cited by 42
Which Percutaneous Tracheostomy Method Is Better? A Systematic Review
Sanabria, Á. (2014) · Respiratory Care · doi:10.4187/respcare.03050 · Cited by 42
Makrygiannakis, M. A., Kaklamanos, E. G., & Athanasiou, A. E. (2018) · European Journal of Orthodontics · doi:10.1093/ejo/cjy001 · Cited by 42
“10 used for the heterogeneity tests (32).”
Kaklamanos, E. G., Makrygiannakis, M. A., & Athanasiou, A. E. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20043537 · Cited by 46
Heterogeneity: the issue of apples, oranges and fruit pie
Purgato, M., & Adams, C. E. (2011) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796011000643 · Cited by 41
“Certainly, perceptions of diversity or its lack may influence meta-analysts on what data to combine, what data to avoid combining, what methods to use to combine and how to interpret the results they eventually get (Ioannidis, 2008).”
BackgroundDrug Resistance Patterns ofEscherichia coliin Ethiopia: A Meta-Analysis
Tuem, K. B., Gebre, A. K., Atey, T. M., Bitew, H., Yimer, E. M., & Berhe, D. F. (2018) · BioMed Research International · doi:10.1155/2018/4536905 · Cited by 41
“The study bias was measured by Begg’s funnel plot [59].”
MethodHepatitis E virus infection among pregnant women in Africa: systematic review and meta-analysis
Dagnew, M., Belachew, A., Tiruneh, M., & Moges, F. (2019) · BMC Infectious Diseases · doi:10.1186/s12879-019-4125-x · Cited by 41
Prevalence of Schistosoma mansoni infection in Ethiopia: a systematic review and meta-analysis
Hussen, S., Fenta, D. A., Tadesse, B. T., & Shimelis, T. (2021) · Tropical Diseases Travel Medicine and Vaccines · doi:10.1186/s40794-020-00127-x · Cited by 41
Measuring Sex Differences and Similarities
Giudice, M. D. (2022) · Focus on sexuality research · doi:10.1007/978-3-030-84273-4_1 · Cited by 43
Papageorgiou, S. N., Konstantinidis, I., Papadopoulou, K., Jäger, A., & Bourauel, C. P. (2014) · Orthodontics and Craniofacial Research · doi:10.1111/ocr.12048 · Cited by 40
“10 used for the heterogeneity tests (53).”
Zenebe, T., Zegeye, N., & Eguale, T. (2020) · Annals of Clinical Microbiology and Antimicrobials · doi:10.1186/s12941-020-00405-8 · Cited by 39
“I2 values of 25%, 50% and 75% are considered to represent low, medium and high heterogeneity respectively [20, 21].”
BackgroundInauen, D., Papadopoulou, A. K., Eliades, T., & Papageorgiou, S. N. (2023) · Clinical Oral Investigations · doi:10.1007/s00784-023-04931-5 · Cited by 40
An analysis of intracranial epidermoid tumors with malignant transformation: treatment and outcomes
Nagasawa, D. T., Choy, W., Spasic, M., Yew, A. Y., Trang, A., Garcia, H. M., & Yang, I. (2012) · Clinical Neurology and Neurosurgery · doi:10.1016/j.clineuro.2012.10.026 · Cited by 39
Childhood PFAS Exposure and Immunotoxicity: A Systematic Review and Meta-Analysis of Human Studies
Antoniou, E. E., & Dekant, W. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02596-z · Cited by 38
“[17].”
Neonatal sepsis and its associated factors in East Africa: a systematic review and meta-analysis
Abate, B. B., Kassie, A. M., Reta, M. A., & Kassaw, M. W. (2020) · International Journal of Public Health · doi:10.1007/s00038-020-01489-x · Cited by 40
“In this study, the I2statistic value of zero indicates true homogeneity, whereas the value 25, 50, and 75% represented low, moderate, and high heterogeneity, respectively (Higgins and Thompson 2002; Ioannidis 2008).”
BackgroundSiemens, W., Schwarzer, G., Rohe, M. S., Buroh, S., Meerpohl, J. J., & Becker, G. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.03.010 · Cited by 39
Dilnessa, T., Getaneh, A., Hailu, W., Moges, F., & Gelaw, B. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262597 · Cited by 38
“I(2) heterogeneity test, in which 0–40%, 50–60%, 50–90%, and 75–100% represented low, moderate, substantial, and considerable heterogeneity, respectively [30, 31].”
BackgroundKitaw, T. A., Abate, B. B., Tilahun, B. D., Yilak, G., Rede, M. B., Getie, ፟., & Haile, R. N. (2024) · BMC Public Health · doi:10.1186/s12889-024-20300-7 · Cited by 37
“Zero invers variance (I 2 ) was revealed a true homogeneity, whereas 25%, 50% and 75% shows a low, moderate and high heterogeneity, respectively [22, 23].”
Background
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- Journal of Evaluation in Clinical Practice
- Publisher
- Wiley
- Published
- 2008-10
- Volume
- 14
- Issue
- 5
- Pages
- 951-957
- ISSN
- 1356-1294 (ISSN-L)
- 1365-2753
- OpenAlex
- W2144112749
- Cited by
- 696
- References
- 57
- Citations by year
- 2026: 52 · 2025: 69 · 2024: 98 · 2023: 72 · 2022: 50 · 2021: 64 · 2020: 49 · 2019: 34 · 2018: 30 · 2017: 23 · 2016: 23 · 2015: 26 · 2014: 27 · 2013: 22 · 2012: 15
- FWCI
- 4.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Evaluation in Clinical Practice (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- heterogeneity testing, publication bias, meta-analysis, statistical heterogeneity, clinical heterogeneity, false positive signals, bias detection
- MeSH terms
- Data Interpretation, Statistical, Humans, Regression Analysis, Research Design, Risk, Sensitivity and Specificity, Meta-Analysis as Topic, Reproducibility of Results, Bias, Confidence Intervals, Publication Bias, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
