Comment on: heterogeneity in meta-analysis should be expected and appropriately quantified
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Higgins1 in his commentary on the study by Patsopoulos and coworkers2 outlines some of the limitations of I2 in assessing heterogeneity in a meta-analysis. These limitations have perhaps not been as explicitly explained before. They have particular implications for assessing heterogeneity in meta-analyses of cohort and database studies and this aspect is worthy of further comment. As pointed out by Higgins, I2 is not a point estimate of the between-study heterogeneity.1 It is the proportion of total variation in the point estimates that is attributable to between-study heterogeneity. The total variation is comprised of between-study variation and within-study variation. If the individual studies have large sample size, such that the within-study variation is small, then I2 could be large (say >75%), even if the between-study variation is small. This is particularly a problem for meta-analyses of cohort and database studies, which typically have large sample sizes. The advantages I2 are that it does not depend on the number of studies in the meta-analysis or the effect measure used (e.g. odds ratio, risk ratio, hazard ratio, mean).3 However, it does depend on the size (precision) of the individual studies.4 It might be sensible to compare I2 across different meta-analyses, where the component randomised controlled trials (RCTs) all have about the same sample size. However, it is not sensible to compare I2 across meta-analyses of RCTs (sample size in the 100s) and meta-analyses of cohort or database studies (sample size in the 1000s).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569
Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified
Higgins, J. P. T. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dyn204 · Cited by 995
Cited by
The global prevalence of common mental disorders: a systematic review and meta-analysis 1980–2013
Steel, Z., Marnane, C., Iranpour, C., Chey, T., Jackson, J. W., Patel, V., & Silove, D. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu038 · Cited by 2,846
Wang, X., Ouyang, Y., Liu, J., Zhu, M., Zhao, G., Bao, W., & Hu, F. B. (2014) · BMJ · doi:10.1136/bmj.g4490 · Cited by 1,596
Winsper, C., Bilgin, A., Thompson, A., Marwaha, S., Chanen, A. M., Singh, S. P., Wang, A., & Furtado, V. (2020) · The British Journal of Psychiatry · doi:10.1192/bjp.2019.166 · Cited by 346
A meta-analysis and review of holistic face processing.
Richler, J. J., & Gauthier, I. (2014) · Psychological Bulletin · doi:10.1037/a0037004 · Cited by 340
Wong, M. C. S., Huang, J., Wang, J., Chan, P. S. F., Lok, V., Chen, X., Leung, C., Wang, H. H. X., Lao, X. Q., & Zheng, Z.-J. (2020) · European Journal of Epidemiology · doi:10.1007/s10654-020-00650-3 · Cited by 327
Jabre, P., Roger, V. L., Murad, M. H., Chamberlain, A. M., Prokop, L., Adnet, F., & Jouven, X. (2011) · Circulation · doi:10.1161/circulationaha.110.986661 · Cited by 314
Taylor, J. B., Goode, A. P., George, S. Z., & Cook, C. E. (2014) · The Spine Journal · doi:10.1016/j.spinee.2014.01.026 · Cited by 291
Yates, K., Lång, U., Cederlöf, M., Boland, F., Taylor, P., Cannon, M., McNicholas, F., DeVylder, J., & Kelleher, I. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.3514 · Cited by 287
Qu, Y., Hu, H.-Y., Ou, Y.-N., Shen, X.-N., Xu, W., Wang, Z.-T., Dong, Q., Tan, L., & Yu, J.-T. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.05.012 · Cited by 240
Adverse childhood experiences and risk of type 2 diabetes: A systematic review and meta-analysis
Huang, H., Yan, P., Shan, Z., Chen, S., Li, M., Luo, C., Gao, H., Hao, L., & Liu, L. (2015) · Metabolism · doi:10.1016/j.metabol.2015.08.019 · Cited by 239
High statistical heterogeneity is more frequent in meta-analysis of continuous than binary outcomes
Alba, A. C., Alexander, P. E., Chang, J., MacIsaac, J., DeFry, S., & Guyatt, G. H. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.09.005 · Cited by 129
“One limitation of I 2 is that if sample sizes are large or study variances are small, unimportant differences can generate large I 2 and potentially excessive concern regarding inconsistency of results [8,9].”
BackgroundYan, D., Jiao, Y., Yan, H., Liu, T., Yan, H., & Yuan, J. (2022) · Environmental Health · doi:10.1186/s12940-022-00858-8 · Cited by 104
Metadata
- Authors
- Coory, M. D.
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-04-06
- Volume
- 39
- Issue
- 3
- Pages
- 932-932
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- OpenAlex
- W2136156766
- Cited by
- 145
- References
- 2
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.