The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Interpretation of tests of heterogeneity and bias in meta-analysis
Abstract
Statistical tests of heterogeneity and bias, in particular publication bias, are very popular in meta-analyses. These tests use statistical approaches whose limitations are often not recognized. Moreover, it is often implied with inappropriate confidence that these tests can provide reliable answers to questions that in essence are not of statistical nature. Statistical heterogeneity is only a correlate of clinical and pragmatic heterogeneity and the correlation may sometimes be weak. Similarly, statistical signals may hint to bias, but seen in isolation they cannot fully prove or disprove bias in general, let alone specific causes of bias, such as publication bias in particular. Both false-positive and false-negative signals of heterogeneity and bias can be common and their prevalence may be anticipated based on some rational considerations. Here I discuss the major common challenges and flaws that emerge in using and interpreting statistical tests of heterogeneity and bias in meta-analyses. I discuss misinterpretations that can occur at the level of statistical inference, clinical/pragmatic inference and specific cause attribution. Suggestions are made on how to avoid these flaws, use these tests properly and learn from them.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These limitations are common in meta-analyses of cross-sectional studies (Ioannidis, 2008).”
From Decision making capacity for treatment in psychiatric inpatients: a systematic review and meta-analysis (2024) · cites this paper as Ioannidis, 2008
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationThe combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Self-citationAssessing heterogeneity in meta-analysis: Q statistic or I² index?
Huedo-Medina, T. B., Sánchez-Meca, J., Marín-Martínez, F., & Botella, J. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.2.193 · Cited by 4,256
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
Cited by
Sorted by citations
Items 301–400 of 695
Zhang, A., Song, Z., Di, A., Zhou, Z., Zheng, L., & Zhuang, L. (2024) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2024.103020 · Cited by 15
Ren, S., Zhao, Y., Wu, J., Ren, S., & Feng, Y. (2024) · Frontiers in Pharmacology · doi:10.3389/fphar.2024.1406588 · Cited by 15
“Statistical heterogeneity was estimated using the I 2 statistic, for which an I 2 value of < 25%, between 26% and 75%, and > 75% represents low, moderate, and high heterogeneity, respectively (Ioannidis, 2008).”
MethodBoltena, M. T., El‐Khatib, Z., Kebede, A. S., Asamoah, B. O., Appiah, S. C. Y., Kamara, K., Assogba, P., Boltena, A. T., Taye, H., Hailemeskel, E., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19095444 · Cited by 16
Parsons, J., Griffiths, S. E., Thomas, N., & Atherton, H. (2021) · Journal of Public Health · doi:10.1093/pubmed/fdab220 · Cited by 15
Microplastics, plastics, and their products exposures and cancer: a pooled analysis
Liu, X., Wang, L., Liu, X., Yu, P., Diao, Y., Chen, T., Sheng, Y., Zhang, D., Peng, D., Guo, Y., et al. (2025) · International Journal of Surgery · doi:10.1097/js9.0000000000002631 · Cited by 15
The Social Work Practice Research Conundrum
Wodarski, J. S. (2011) · Journal of Human Behavior in the Social Environment · doi:10.1080/10911359.2011.580249 · Cited by 14
“This is also in addition to potential publication bias; often the studies submitted and chosen for publication are ones that yielded significant positive findings (Ioannidis, 2008).”
BackgroundMethodInfluentialHou, L., Peng, B., Zhang, D., Yang, J., Wang, Y., Li, T., Li, S., Wang, Q., & Zhao, J. (2021) · Frontiers in Neurology · doi:10.3389/fneur.2021.712717 · Cited by 14
“10) of heterogeneity (16); otherwise, the fixed-effects model was used.”
MethodGeng, C., Huang, L., Pu, L., & Feng, Y. (2023) · Frontiers in Nutrition · doi:10.3389/fnut.2022.1001826 · Cited by 14
Wang, C., Sheng, Y., Yu, L., Tian, F., Xue, Y., & Zhai, Q. (2023) · Behavioural Brain Research · doi:10.1016/j.bbr.2023.114653 · Cited by 14
Alemu, T. G., Tamir, T. T., Workneh, B. S., Mekonen, E. G., Ali, M. S., Zegeye, A. F., Wassie, M., Kassie, A. T., Tekeba, B., & Gonete, A. T. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1359572 · Cited by 14
“The studies’ heterogeneity was measured using the I-squared statistic; An I -squared statistic of 25, 50, and 75%, respectively, indicated low, moderate, and high heterogeneity (22, 23).”
MethodAbate, B. B., Kumsa, H., Kibret, G. A., Wodaynew, T., Habtie, T. E., Mezen, M. K., Munie, M. A., Temesgen, D., Tilahun, B. D., Faris, A. M., et al. (2024) · Neuroepidemiology · doi:10.1159/000539803 · Cited by 14
“In this study, the I 2 statistic value of zero indicates true homogeneity, whereas 25%, 50%, and 75% represent low, moderate, and high heterogeneity, respectively [30, 38].”
Kassa, Z. Y., Hussen, S., Hadra, N., Mittiku, Y. M., & Bonja, F. (2020) · The European Journal of Contraception & Reproductive Health Care · doi:10.1080/13625187.2020.1779688 · Cited by 14
Effect of enhanced recovery after surgery on inflammatory bowel disease surgery: A meta-analysis
Peng, D., Cheng, Y.-X., Tao, W., Tang, H., & Ji, G. (2022) · World Journal of Clinical Cases · doi:10.12998/wjcc.v10.i11.3426 · Cited by 14
A test for reporting bias in trial networks: simulation and case studies
Trinquart, L., Ioannidis, J. P. A., Châtellier, G., & Ravaud, P. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-112 · Cited by 15
“In the absence of significant signals of bias, one should not exclude the possibility of reporting bias, because a âĂIJnegativeâĂİ test result typically does not greatly decrease the prior odds of bias, unless in cases of many trials or substantial between-trial heterogeneity [47].”
BackgroundSelf-citationMartín-Saavedra, J. S., & Ruiz-Sternberg, Á. (2020) · Nordic Journal of Music Therapy · doi:10.1080/08098131.2020.1761867 · Cited by 12
Seeing the forest for the trees: How to interpret a meta‐analysis forest plot
Alavi, M., Hunt, G. E., Visentin, D. C., Watson, R. A., Thapa, D. K., & Cleary, M. L. (2020) · Journal of Advanced Nursing · doi:10.1111/jan.14721 · Cited by 14
Ioannidis, J. P. A., & Contopoulos‐Ioannidis, D. G. (2023) · International Journal of Infectious Diseases · doi:10.1016/j.ijid.2023.06.009 · Cited by 13
“Heterogeneity was assessed with he chi-square-based Q test and with the I 2 statistic [5] .”
Self-citationRen, S., Yao, X., Ren, S., & Feng, Y. (2023) · Frontiers in Nutrition · doi:10.3389/fnut.2023.1294064 · Cited by 13
“Statistical heterogeneity was estimated using the I 2 statistic (11).”
BackgroundMethodWu, Q., Wang, Y., Peng, Q., Bai, M., Shang, Z., Li, L., Tian, F., & Jing, C. (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000921 · Cited by 13
“Statistical signi fi cance was recognized when the forest plot P -value was 0.05 [10] .”
BackgroundInfluentialLi, X., Fang, L., Guan, L., Zhang, J., Zheng, M., & Zhu, D. (2023) · Brain and Behavior · doi:10.1002/brb3.3339 · Cited by 14
“I 2 values of 25% were classified as low heterogeneity, 50% as moderate heterogeneity, and 75% as highheterogeneity(Ioannidis,2008).”
BackgroundLi, Y., Fang, W., Qiu, H., Dong, W., Wang, J., & Bao, H. (2023) · LEUKOS The Journal of the Illuminating Engineering Society of North America · doi:10.1080/15502724.2023.2279946 · Cited by 16
“Heterogeneity was quantified by calculating the I 2 statistics with 25%, 50%, and 75% indicating low, moderate, and high levels of heterogeneity, respectively (Higgins et al. 2003), which was considered preferable to test the consistency of evidence (Ioannidis 2008).”
MethodDawood, Z. S., Brown, Z. J., Alaimo, L., Lima, H. A., Shaikh, C. F., Katayama, E., Munir, M. M., Moazzam, Z., Endo, Y., Woldesenbet, S., et al. (2024) · HPB · doi:10.1016/j.hpb.2024.02.003 · Cited by 13
Outcomes of liver transplant for colorectal liver metastasis: a systematic review and meta-analysis
Dawood, Z. S., Brown, Z. J., Munir, M. M., Waqar, U., Rawicz‐Pruszyński, K., Endo, Y., Gajjar, A., Schenk, A. D., & Pawlik, T. M. (2024) · Journal of Gastrointestinal Surgery · doi:10.1016/j.gassur.2024.09.006 · Cited by 13
Tusa, B. S., Alati, R., Ayano, G., Betts, K. S., Weldesenbet, A. B., & Dachew, B. A. (2024) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2024.104261 · Cited by 13
Liu, Z., Liu, B., Yu, H., Zhang, H., He, Z., & Zhuo, Z. (2024) · Insects · doi:10.3390/insects15110888 · Cited by 13
“Potential moderators considered included the effects of humidity and photoperiod on cumulative effect size [33].”
Zhou, P., Xie, Y., Yan, Z., Liu, X., & Hua, H. (2019) · Bioscience Reports · doi:10.1042/bsr20191519 · Cited by 14
Wang, H., Wang, D., Song, H., Zou, D., Feng, X., Ma, X., Miao, J., Yang, W., & Wang, H. (2021) · Materials Express · doi:10.1166/mex.2021.1888 · Cited by 13
Li, Z., Tong, Y. A., Liu, F., Liu, X., Lv, Q., Tang, K.-L., Li, L., Liu, X.-Y., Zhang, W., & Peng, D. (2023) · Langenbeck s Archives of Surgery · doi:10.1007/s00423-023-03171-1 · Cited by 12
“The fixed effects model was used for this study with I 2 ≤ 50%, and P 0.05, which was considered statistically significant [13].”
MethodInfluentialResting-state functional abnormalities in ischemic stroke: a meta-analysis of fMRI studies
Zhang, Z. (2024) · Brain Imaging and Behavior · doi:10.1007/s11682-024-00919-1 · Cited by 13
“These inconsistent findings may be due to small sample sizes, divergent age ranges, different statistical methods, and differences in patient samples (Ioannidis, 2008).”
Beressa, G., Whiting, S. J., Kuma, M. N., Lencha, B., & Belachew, T. (2024) · PLoS ONE · doi:10.1371/journal.pone.0310329 · Cited by 22
Human papillomavirus vaccine uptake and its determinants among women in Africa: an umbrella review
Mengistie, B. A., Yirsaw, A. N., Lakew, G., Mekonnen, G. B., Shibabaw, A. A., Chereka, A. A., Kitil, G. W., Wondie, W. T., Abuhay, A. E., & Getachew, E. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1537250 · Cited by 12
Tao, W. Y., Liu, X., Cheng, Y.-X., Kang, B., Zhang, H., Yuan, C., Zhang, B., & Peng, D. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2021.715040 · Cited by 12
“The I(2) value and the results of the chi-squared test were used to assess the statistical heterogeneity (24, 25).”
MethodLee, Y., Yoon, J. W., Kim, Y. A., Choi, H. J., Yoon, B. W., & Seo, J. H. (2022) · Genes · doi:10.3390/genes13091553 · Cited by 13
“05 indicates heterogeneity between the two datasets [33].”
BackgroundLiu, F., Wang, L., Liu, X., Li, Z., & Peng, D. (2023) · Journal of Laparoendoscopic & Advanced Surgical Techniques · doi:10.1089/lap.2023.0119 · Cited by 12
“31,32 We used the random effects model, and P < .1 was considered statistically significant.”
Leung, L., Han, H., Martin, M. C., & Kotecha, J. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-007650 · Cited by 12
Zenebe, T., Mitiku, M., & Alem, Y. (2020) · International Journal of Microbiology · doi:10.1155/2020/8844294 · Cited by 12
“I(2) values of 25%, 50%, and 75% are considered to represent low, medium, and high heterogeneity, respectively [27, 28].”
BackgroundMohammed, A. O. H. A., & Kaklamanos, E. G. (2021) · European Journal of Orthodontics · doi:10.1093/ejo/cjab013 · Cited by 12
Reflections on the I-squared index for measuring inconsistency in meta-analysis
Higgins, J. P. T., & López-López, J. A. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10062 · Cited by 14
“13 Although 𝐼 2 has been welcomed by some, 14,15 its popularity has been accompanied by some common misconceptions and misinterpretations that hamper its usefulness in practice.”
Lin, Y., Xu, G., Li, L., Xiang, J., & Zhai, L. (2024) · Medicine · doi:10.1097/md.0000000000038018 · Cited by 11
“[29,30] 3”
MethodChao, P., Chang, C.-H., Wu, J.-J., Liu, Y.-H., Shiau, J., Shih, H.-H., Lin, G.-Z., Lee, S., & Lee, T.-F. (2024) · Cancer Control · doi:10.1177/10732748241286749 · Cited by 11
“8 At least two independent reviewers are required to minimize the in fl uence of individual bias in bias risk assessment.”
Overoptimism in the interpretation of statistics
Poole, D., Nattino, G., & Bertolini, G. (2014) · Intensive Care Medicine · doi:10.1007/s00134-014-3510-6 · Cited by 11
He, Q., Zhang, Q., & Zhang, D. (2016) · Digestive Diseases and Sciences · doi:10.1007/s10620-016-4291-2 · Cited by 11
“Funnel plots were used to identify the potential presence of publication bias, and Begg’s or Egger’s weighted regression statistics were used when necessary [23].”
MethodWang, J., Chen, X., Zhao, J., & Zhang, W. W. (2022) · Annals of Vascular Surgery · doi:10.1016/j.avsg.2022.04.039 · Cited by 11
Ruvalcaba, Y., Rodríguez, A. L., Eaton, A. A., Stephens, D. P., & Madhivanan, P. (2022) · Aggression and Violent Behavior · doi:10.1016/j.avb.2022.101760 · Cited by 10
Liao, G., Wang, Z., Zhang, Y., Qian, K., Mac, S. M., Li, H., & Huang, Z. (2020) · Medicine · doi:10.1097/md.0000000000019225 · Cited by 11
“1 and an I(2) value of greater than 50% was considered statistically significant for high heterogeneity.([19,20]) The fixedeffect model was used when no high heterogeneity was detected among the studies, while the random-effect model was preferred for the studies with high statistical heterogeneity.”
MethodPabalan, N., Chaisri, S., Tabunhan, S., Tarasuk, M., Jarjanazi, H., & Steiner, T. S. (2017) · Acta Tropica · doi:10.1016/j.actatropica.2017.05.007 · Cited by 11
Heterogeneity and clinical reality
Knottnerus, J. A., & Tugwell, P. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.05.009 · Cited by 11
Wen, Z., Xiao, D., & Zhou, X. (2022) · Frontiers in Surgery · doi:10.3389/fsurg.2022.924444 · Cited by 10
“05 was considered statistically significant (24, 25).”
Liu, F., Tang, S.-J., Li, Z., Liu, X., Lv, Q., Zhang, W., & Peng, D. (2024) · World Journal of Gastrointestinal Surgery · doi:10.4240/wjgs.v16.i2.585 · Cited by 10
“I 2 values and χ 2 tests were used to assess the statistical heterogeneity[25,26].”
Weissfeld, A. S., Baselski, V. S., Cornish, N. E., Kraft, C. S., LaRocco, M., McNult, P., Nachamkin, I., Parrott, J. S., Richter, S. S., Rubinstein, M. L., et al. (2024) · Clinical Microbiology Reviews · doi:10.1128/cmr.00065-18 · Cited by 11
Abate, B. B., Tusa, B. S., Sendekie, A. K., Araya, F. G., Azmeraw, M., Walle, G. T., Kitaw, T. A., Tilahun, B. D., Alamaw, A. W., Zemariam, A. B., et al. (2025) · International Breastfeeding Journal · doi:10.1186/s13006-025-00712-w · Cited by 17
“An I² value of zero indicated homogeneity, while values of 25%, 50%, and 75% were considered low, moderate, and high heterogeneity, respectively [32, 33].”
Antonoglou, G. N., Papageorgiou, S. N., de Albornoz, A. C., Payer, M., & Stavropoulos, A. (2025) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.70021 · Cited by 10
Sero-prevalence of rubella among pregnant women in Sub-Saharan Africa: a meta-analysis
Kassa, Z. Y., Hussen, S., & Asnake, S. (2020) · Human Vaccines & Immunotherapeutics · doi:10.1080/21645515.2020.1729027 · Cited by 10
Baers, J. H., Adekanye, J., Hazlewood, G. S., Davies, J. M., Caird, J. K., & Wilton, S. B. (2022) · Reviews in Cardiovascular Medicine · doi:10.31083/j.rcm2310353 · Cited by 10
“Due to the very significant heterogeneity in the design and implementation of the individual patient decision aids, as well as the interventions compared in each study, a quantitative meta-analysis was not performed, in accordance with best practices in systematic reviews [27,28].”
MethodBrouwer, J., van den Berg, F., Knooihuizen, R., Loerts, H., & Keijzer, M. C. J. (2020) · Behavioral Sciences · doi:10.3390/bs10090132 · Cited by 11
“Since the present study dealt with a small number of studies per comparison, heterogeneity was to some extent expected [136].”
BackgroundTooth wear during orthodontic treatment with fixed appliances: a systematic review
Makrygiannakis, M. A., Kaklamanos, E. G., Milosevic, A., & Athanasiou, A. E. (2018) · Journal of Orthodontics · doi:10.1080/14653125.2018.1517469 · Cited by 10
Paphiti, A., & Newman, E. (2023) · International Journal of Cognitive Therapy · doi:10.1007/s41811-023-00184-y · Cited by 11
“Heterogeneity poses a challenge to interpreting meta-analysis (Ioannidis, 2008) and where meta-analysis is not seen as appropriate, alternative approaches to synthesising results should be explored (McKenzie & Brennan, 2019).”
Hadebe, M. I., Manyangadze, T., Kalinda, C., Mindu, T., & Chimbari, M. J. (2023) · Tropical Medicine and Infectious Disease · doi:10.3390/tropicalmed8100467 · Cited by 9
“Additionally, to explain the observed heterogeneity, subgroup analysis was performed using our data stratified by snail species and the African regions where the studies were conducted [23,24].”
MethodWang, J., Luan, F., Bai, Z., & Liu, Z. (2024) · Updates in Surgery · doi:10.1007/s13304-024-01821-7 · Cited by 9
“Interstudy heterogeneity was assessed using the Cochran Q statistic ( χ 2 test) [17], along with the I 2 statistic as a measure describing degrees of heterogeneity [18].”
Gebrehana, D. A., Molla, G. E., Endalew, W., Teshome, D. F., Mekonnen, F. A., & Angaw, D. A. (2024) · BMC Infectious Diseases · doi:10.1186/s12879-024-09926-5 · Cited by 10
“After the asymmetric funnel plot was found, publication bias was assessed using the Egger weighted regression test ( P < 0.05) [24], and this bias was handled by trim and fill analysis [25].”
MethodInfluentialLi, Y., Tian, M., Zhong, W., Zou, J., Duan, X., & Si, H. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1552271 · Cited by 9
“…blood transfusion (defined as ten or more units of red blood cells within 24 h or more than 40 ml/kg PRBC in 2 h or less) was significantly higher in the placebo group than in the TXA group, suggesting that prehospital TXA is associated with a significant reduction in transfusion requirements (22).”
InfluentialSaunders, B., & Inzlicht, M. (2021) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2021.01.018 · Cited by 9
“However, as has been noted elsewhere (Ioannidis, 2008), this method lacks objectivity.”
MethodAbnormalities of regional spontaneous brain activity in poststroke aphasia: a meta-analysis
Cheng, L. L., Xi, H., Gu, H., Gao, Y., Hu, S., Li, M., Hao, Z., Wen, J., Zhang, J., Lv, Y., et al. (2023) · Cerebral Cortex · doi:10.1093/cercor/bhad078 · Cited by 9
Kongjam, P., Pabalan, N., Tharabenjasin, P., Jarjanazi, H., Chai่jaroenkul, W., & Na‐Bangchang, K. (2023) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0011735 · Cited by 9
Cheng, Y.-X., Tao, W. Y., Liu, X., Zhang, H., Yuan, C., Zhang, B., Zhang, W., & Peng, D. (2021) · Nutrition and Cancer · doi:10.1080/01635581.2021.1993277 · Cited by 9
“1 was considered statistically significant (21).”
MethodInfluentialProblems Caused by Heterogeneity in Meta-Analysis: A Case Study of Acupuncture Trials
Prady, S. L., Burch, J., Crouch, S., & MacPherson, H. (2013) · Acupuncture in Medicine · doi:10.1136/acupmed-2013-010364 · Cited by 9
Evidence Synthesis: Evolving Methodologies to Optimise Patient Care and Enhance Policy Decisions
Ashrafian, H., Darzi, A. W., & Athanasiou, T. (2011) · Springer eBooks · doi:10.1007/978-0-85729-206-3_1 · Cited by 9
Wu, Z., Lai, J., Huang, Q., Lin, L., Lin, S., Chen, X., & Huang, Y. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1285904 · Cited by 8
“Furthermore, meta-regression and subgroup analyses were carried out to explore potential sources of heterogeneity among studies (Ioannidis, 2008).”
MethodDesta, D. T., Teferra, T. F., & Gebremedhin, S. (2024) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph21101269 · Cited by 8
“The values indicate that while the overall between-study variance might be low, there are likely substantial differences between specific subgroups of studies, requiring subgroup analysis [73].”
BackgroundResultLiu, X., Zhao, Z., Cheng, Y.-X., Tao, W. Y., Yuan, C., Zhang, B., & Wang, C. (2022) · Frontiers in Surgery · doi:10.3389/fsurg.2022.870946 · Cited by 8
“The I2 value and the results of the chi-squared test were used to assess statistical heterogeneity (20, 21).”
MethodNwoko, O. E., Kalinda, C., & Chimbari, M. J. (2022) · Tropical Medicine and Infectious Disease · doi:10.3390/tropicalmed7050072 · Cited by 8
“Furthermore, subgroup analysis was carried out by stratifying our data by snail species and the countries where the studies were conducted to potentially explain the observed heterogeneity [16,17].”
MethodBarrit, S., Hadwe, S. E., Barajraji, M. A., Torcida, N., Bogossian, E. G., André, J., Niset, A., Carron, R., Taccone, F. S., & Madsen, J. R. (2023) · Neurocritical Care · doi:10.1007/s12028-023-01885-0 · Cited by 9
“We generated funnel plots to detect bias or systematic heterogeneity between included studies visually; however, their use has significant limitations [12].”
Statistical heterogeneity in meta-analysis of hypertension and exercise training: A meta-review
Teixeira, J. M. M., Roever, L., Ramasamy, A., Pereira, R., Carneiro, I., Krustrup, P., & Póvoas, S. C. A. (2023) · Journal of Sports Sciences · doi:10.1080/02640414.2024.2309055 · Cited by 8
“T. Higgins, 2003; J. P. T. Higgins & Thompson, 2002; J. P. T. Higgins et al., 2019; J. P. A. Ioannidis, 2008).”
BackgroundBisphosphonates as Supplement to Dental Treatment: A Network Meta-Analysis
Zymperdikas, V. F., Yavropoulou, M. P., Kaklamanos, E. G., & Papadopoulos, M. A. A. P. (2020) · Journal of Dental Research · doi:10.1177/0022034520972945 · Cited by 8
“Possible sources of heterogeneity were planned to be searched through predetermined mixed-effects subgroup analyses and random-effects meta-regression to identify any possible factors that could influence the treatment results (Ioannidis 2008).”
MethodErectile dysfunction among testicular cancer survivors: A systematic review and meta-analysis
Kerie, S., Workineh, Y., Kasa, A. S., Ayalew, E., & Menberu, M. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e07479 · Cited by 8
Carter, T. J., Pandey, G., Bolger, N. P., Hassin, R. R., & Ferguson, M. J. (2020) · Social Cognition · doi:10.1521/soco.2020.38.6.489 · Cited by 8
Hadwe, S. E., Assamadi, M., Barrit, S., Giannis, D., Haidich, A., Goulis, D. G., & Chatzisotiriou, A. (2022) · Brain Injury · doi:10.1080/02699052.2022.2065034 · Cited by 8
“McCammack, 2015 144 134 134 100% 74 (20) 77(53.”
BackgroundThe prognostic value of B7H1 and B7H4 expression in pancreatic cancer: A meta-analysis
Qi, Z.-J., Yu, D., Chen, C., Jiang, H., Li, R., & Kang, Y. (2019) · The International Journal of Biological Markers · doi:10.1177/1724600819881147 · Cited by 8
Coyne, J., Bhar, S., Pignotti, M., Tovote, K., & Beck, A. (2010) · Psychotherapy and Psychosomatics · doi:10.1159/000317537 · Cited by 8
Abate, B. B., Kassie, A. M., Kassaw, M. W., & Reta, M. A. (2019) · doi:10.21203/rs.2.16426/v1 · Cited by 8
Prone position for acute respiratory distress syndrome and the hazards of meta-analysis
Poole, D., Pisa, A., & Fumagalli, R. (2023) · Pulmonology · doi:10.1016/j.pulmoe.2022.12.005 · Cited by 8
“42 TaggedEnd TaggedP Our study supports the hypothesis that the PROSEVA trial is an outlier compared to the RCTs that preceded it.”
Endeshaw, D., Dessie, G., Kasa, A. S., Endalamaw, A., Tesfaye, T. D., Birhanu, S., Abate, T. W., Afewerk, S., Bogale, E. K., & Workineh, Y. (2024) · Scientific Reports · doi:10.1038/s41598-024-56574-5 · Cited by 7
“To assess the quality of each study, we utilized The Newcastle–Ottawa Scale 13 , which was adapted for the systematic review to evaluate cross-sectional studies 14 .”
Gupta, B., Gupta, K., Goel, R., & Mittal, A. (2024) · National Journal of Community Medicine · doi:10.55489/njcm.150920244162 · Cited by 7
Factors associated with contracting border malaria: A systematic and meta-analysis
Fambirai, T., Chimbari, M. J., & Mindu, T. (2025) · PLoS ONE · doi:10.1371/journal.pone.0310063 · Cited by 7
Piñar-Lara, M., Obrero‐Gaitán, E., Lomas‐Vega, R., López‐Ruiz, M. D. C., García‐López, H., & Cortés‐Pérez, I. (2025) · Disability and Rehabilitation · doi:10.1080/09638288.2025.2458186 · Cited by 7
“[60–62].”
Qiu, Q., Thoo, A. C., Sarker, M., & Huam, H. T. (2025) · Transportation Planning and Technology · doi:10.1080/03081060.2025.2493920 · Cited by 7
Adisu, M. A., Zemariam, A. B., Derso, Y. A., Molla, B., Kassa, M. A., Kitaw, T. A., & Habtie, T. E. (2025) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2025.110544 · Cited by 7
Thyroid nodule size and risk of malignancy: a systematic review
Cotter, A., & Jinih, M. (2025) · Discover Oncology · doi:10.1007/s12672-025-02588-y · Cited by 13
Wang, L., Liu, X., Xiang, C., Shu, X., Liu, X., Zhao, J., Li, S., Xu, R., Chen, B., Kang, B., et al. (2026) · Ecotoxicology and Environmental Safety · doi:10.1016/j.ecoenv.2026.119742 · Cited by 7
Liu, X., Zhang, B., Cheng, Y.-X., Tao, W. Y., Yuan, C., Wei, Z., & Peng, D. (2022) · World Journal of Gastrointestinal Oncology · doi:10.4251/wjgo.v14.i6.1199 · Cited by 7
Ortiz, H. A. D. L. B., Jélvez, F., Parraguez, D., Pérez, F., & Vargas, C. (2023) · Advances in Rehabilitation · doi:10.5114/areh.2023.131577 · Cited by 7
“The Mantel-Haenszel fixed effects method was used to determine the pooled effect by mean difference (MD) [23,24].”
BackgroundMethodInfluentialYarlagadda, S., Selvakumar, R. J., Parashar, S. R., Arockiam, S., & Natanasabapathy, V. (2023) · PubMed Central · doi:10.4103/jcd.jcd_305_23 · Cited by 7
“[14] All P values were two-sided with α =5%, except for the test of between-study heterogeneity ( α =10%).”
BackgroundR, V., Singhal, R., Namdev, R., Kumar, A., Dayma, C., & Rani, A. (2024) · The Cleft Palate-Craniofacial Journal · doi:10.1177/10556656241249822 · Cited by 6
Yilak, G., Tilahun, B. D., Abate, B. B., Zemariam, A. B., Lake, E. S., Ayele, M., & Belay, A. S. (2024) · PLoS ONE · doi:10.1371/journal.pone.0303749 · Cited by 7
López-Vivancos, A., González-Gálvez, N., Orquín-Castrillón, F. J., Vale, R. G. D. S., & Marcos-Pardo, P. J. (2023) · Applied Sciences · doi:10.3390/app13085203 · Cited by 9
Hull, R. D., Liang, J. J., & Brant, R. (2010) · Thrombosis Research · doi:10.1016/j.thromres.2010.05.009 · Cited by 7
Abate, B. B., Kassie, A. M., Kassaw, M. W., & Getu, M. A. (2019) · Research Square · doi:10.21203/rs.2.13740/v1 · Cited by 8
The role of philosophy and ethics at the edges of medicine
Hofmann, B. (2021) · Philosophy Ethics and Humanities in Medicine · doi:10.1186/s13010-021-00114-w · Cited by 6
“A wide range of assumptions are made in evidence production that are not true [85], not relevant”
BackgroundAntoniou, E. E., Rooseboom, M., Kocabaş, N. A., North, C. M., & Zeegers, M. P. (2023) · Mutation Research/Genetic Toxicology and Environmental Mutagenesis · doi:10.1016/j.mrgentox.2023.503689 · Cited by 6
Abuhay, H. W., Derseh, N. M., Kolbe-Alexander, T. L., Gyawali, P., & Yenit, M. K. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-084073 · Cited by 6
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- Journal of Evaluation in Clinical Practice
- Publisher
- Wiley
- Published
- 2008-10
- Volume
- 14
- Issue
- 5
- Pages
- 951-957
- ISSN
- 1356-1294 (ISSN-L)
- 1365-2753
- OpenAlex
- W2144112749
- Cited by
- 696
- References
- 57
- Citations by year
- 2026: 52 · 2025: 69 · 2024: 98 · 2023: 72 · 2022: 50 · 2021: 64 · 2020: 49 · 2019: 34 · 2018: 30 · 2017: 23 · 2016: 23 · 2015: 26 · 2014: 27 · 2013: 22 · 2012: 15
- FWCI
- 4.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Evaluation in Clinical Practice (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- heterogeneity testing, publication bias, meta-analysis, statistical heterogeneity, clinical heterogeneity, false positive signals, bias detection
- MeSH terms
- Data Interpretation, Statistical, Humans, Regression Analysis, Research Design, Risk, Sensitivity and Specificity, Meta-Analysis as Topic, Reproducibility of Results, Bias, Confidence Intervals, Publication Bias, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
