The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Evaluation of Old and New Tests of Heterogeneity in Epidemiologic Meta-Analysis
Open accessAbstract
The identification of heterogeneity in effects between studies is a key issue in meta-analyses of observational studies, since it is critical for determining whether it is appropriate to pool the individual results into one summary measure. The result of a hypothesis test is often used as the decision criterion. In this paper, the authors use a large simulation study patterned from the key features of five published epidemiologic meta-analyses to investigate the type I error and statistical power of five previously proposed asymptotic homogeneity tests, a parametric bootstrap version of each of the tests, and tau2-bootstrap, a test proposed by the authors. The results show that the asymptotic DerSimonian and Laird Q statistic and the bootstrap versions of the other tests give the correct type I error under the null hypothesis but that all of the tests considered have low statistical power, especially when the number of studies included in the meta-analysis is small (<20). From the point of view of validity, power, and computational ease, the Q statistic is clearly the best choice. The authors found that the performance of all of the tests considered did not depend appreciably upon the value of the pooled odds ratio, both for size and for power. Because tests for heterogeneity will often be underpowered, random effects models can be used routinely, and heterogeneity can be quantified by means of R(I), the proportion of the total variance of the pooled effect measure due to between-study variance, and CV(B), the between-study coefficient of variation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These measures should be interpreted with caution: the I2 statistic is captured with uncertainty [73], and Cochrane’s Q test has limited power to detect heterogeneity when the number of included studies is low [74].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [74]
In the citing paper
“This approach was, however, often criticised because of low power to detect heterogeneity in case of few (< 10) included studies [52, 77, 101, 104, 116, 118].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [104]
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 4×
“The most popular method of pooling results was the use of DerSimonian and Laird's random effects model (10).”
BackgroundMethodInfluentialStatistical Aspects of the Analysis of Data From Retrospective Studies of Disease
(1959) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/22.4.719 · Cited by 14,689 · cited 2×
“Mantel and Haenszel (8) and Yusuf et al.”
MethodThe combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380 · cited 1×
“The heterogeneity test described by DerSimonian and Laird (10) was previously proposed and discussed by Cochran (11).”
Beta blockade during and after myocardial infarction: An overview of the randomized trials
Yusuf, S., Peto, R., Lewis, J., Collins, R., & Sleight, P. (1985) · Progress in Cardiovascular Diseases · doi:10.1016/s0033-0620(85)80003-7 · Cited by 3,205 · cited 1×
“(9) have proposed heterogeneity tests that require the raw cell counts and/or do not generalize to a multivariate setting.”
BackgroundReview papers : The statistical basis of meta-analysis
Fleiss, J. (1993) · Statistical Methods in Medical Research · doi:10.1177/096228029300200202 · Cited by 1,281
Variance components testing in the longitudinal mixed effects model
Stram, D. O., & Lee, J. W. (1994) · Biometrics · doi:10.2307/2533455 · Cited by 992 · cited 1×
“The application of a likelihood ratio test (LRT) statistic to heterogeneity problems has been described by Stram and Lee (13).”
MethodA random‐effects regression model for meta‐analysis
Berkey, C. S., Hoaglin, D. C., Mosteller, F., & Colditz, G. A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780140406 · Cited by 879 · cited 1×
“In either the possibly underpowered case or the overpowered case, heterogeneity can be jointly quantified by /?, and CVB, and mixed effects regression models can be used to investigate and adjust for identifiable sources of heterogeneity when Rt and CVfl indicate that the magnitude of the between-study heterogeneity i…”
BackgroundDiabetes mellitus as a risk factor for pancreatic cancer. A meta-analysis
Everhart, J. (1995) · JAMA · doi:10.1001/jama.273.20.1605 · Cited by 742
Mortimer, J. A., Van Duijn, C. M., Chandra, V, Fratiglioni, L., Graves, A. B., Heyman, A., Jorm, A. F., Kokmen, E., Kondo, K., Rocca, W. A., et al. (1991) · International Journal of Epidemiology · doi:10.1093/ije/20.supplement_2.s28 · Cited by 594
Cited by
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
“[12], who present a statistic in the form of I 2 as a quanti cation of heterogeneity in a meta-analysis.”
BackgroundBody-mass index and mortality among 1.46 million white adults
Berrington de Gonzalez, A., Hartge, P., Cerhan, J. R., Flint, A. J., Hannan, L., MacInnis, R. J., Moore, S. C., Tobias, G. S., Anton-Culver, H., Freeman, L. B., et al. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa1000367 · Cited by 2,317
Association of Leisure-Time Physical Activity With Risk of 26 Types of Cancer in 1.44 Million Adults
Moore, S. C., Lee, I.-M., Weiderpass, E., Campbell, P. T., Sampson, J. N., Kitahara, C. M., Keadle, S. K., Arem, H., Berrington de Gonzalez, A., Hartge, P., et al. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.1548 · Cited by 1,487
“44 million participants (median [range] age, 59 [19-98] years; 57%”
Worldwide burden of HIV in transgender women: a systematic review and meta-analysis.
Baral, S. D., Poteat, T., Strömdahl, S., Wirtz, A. L., Guadamuz, T. E., & Beyrer, C. (2013) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(12)70315-8 · Cited by 1,461
Cheung, M. W.-L. (2014) · Psychological Methods · doi:10.1037/a0032968 · Cited by 1,342
“Takkouche, Cadarso-Suárez, and Spiegelman (1999) suggested using the harmonic mean of vi, ṽHM n i 1 n 1 ⁄ vi , (13) whereas Higgins and Thompson (2002) proposed defining “typical” within-study variance using the Q statistic via ṽQ (n 1) i 1 n 1 ⁄ vi i 1 n 1 ⁄ vi 2 i 1 n 1 ⁄ vi 2 .”
Method
Metadata
- Authors
- Takkouche, B., Cadarso-Suarez, C., & Spiegelman, D.
- Affiliations
- Universidade de Santiago de Compostela (ES)
- Harvard University (US)
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 1999-07-15
- Volume
- 150
- Issue
- 2
- Pages
- 206-215
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- OpenAlex
- W2049217505
- Cited by
- 380
- References
- 20
- Citations by year
- 2026: 5 · 2025: 14 · 2024: 15 · 2023: 13 · 2022: 14 · 2021: 18 · 2020: 19 · 2019: 13 · 2018: 13 · 2017: 16 · 2016: 16 · 2015: 18 · 2014: 20 · 2013: 19 · 2012: 20
- FWCI
- 11.62
- Type
- journal article
- Language
- en
- Open access
- green
- nrs.harvard.edu
- Licence
- cc-by
- Locations
- American Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- Xunta de Galicia
- Universidade de Santiago de Compostela
- National Institutes of Health
- National Cancer Institute (CA-55075)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- heterogeneity testing, meta-analysis, parametric bootstrap, type I error, statistical power, random effects models, between-study variance, coefficient of variation, pooled odds ratio, simulation study
- MeSH terms
- Epidemiologic Methods, Humans, Meta-Analysis as Topic, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluation of Old and New Tests of Heterogeneity in Epidemiologic Meta-Analysis. https://app.cochat.ai/papers/10.1093/oxfordjournals.aje.a009981
- The paper
- Takkouche, B., Cadarso-Suarez, C., & Spiegelman, D. (1999). Evaluation of Old and New Tests of Heterogeneity in Epidemiologic Meta-Analysis. American Journal of Epidemiology. https://doi.org/10.1093/oxfordjournals.aje.a009981
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.