The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Confidence intervals for the amount of heterogeneity in meta‐analysis
Abstract
Effect size estimates to be combined in a systematic review are often found to be more variable than one would expect based on sampling differences alone. This is usually interpreted as evidence that the effect sizes are heterogeneous. A random‐effects model is then often used to account for the heterogeneity in the effect sizes. A novel method for constructing confidence intervals for the amount of heterogeneity in the effect sizes is proposed that guarantees nominal coverage probabilities even in small samples when model assumptions are satisfied. A variety of existing approaches for constructing such confidence intervals are summarized and the various methods are applied to an example to illustrate their use. A simulation study reveals that the newly proposed method yields the most accurate coverage probabilities under conditions more analogous to practice, where assumptions about normally distributed effect size estimates and known sampling variances only hold asymptotically. Copyright © 2006 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Linear Statistical Inference and its Applications: Second Editon
Rao, C. R. (1973) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316436 · Cited by 10,489
Self, S. G., & Liang, K.-Y. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478472 · Cited by 2,661
(2000) · Queen Mary Research Online (Queen Mary University of London) · Cited by 1,795
Bias and Efficiency of Meta-Analytic Variance Estimators in the Random-Effects Model
Viechtbauer, W. (2005) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986030003261 · Cited by 1,699
Self-citationAdvanced methods in meta‐analysis: multivariate approach and meta‐regression
van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1040 · Cited by 1,580
Parametric empirical Bayes inference: theory and applications
Morris, C. N. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477920 · Cited by 1,448
Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated
Thompson, S. G. (1994) · BMJ · doi:10.1136/bmj.309.6965.1351 · Cited by 1,017 · cited 2×
“However, it may be more important to actually quantify the extent of the heterogeneity than to rely on an overall statistical test to detect its presence [2].”
BackgroundVariance components testing in the longitudinal mixed effects model
Stram, D. O., & Lee, J. W. (1994) · Biometrics · doi:10.2307/2533455 · Cited by 992
A comparison of statistical methods for meta‐analysis
Brockwell, S. E., & Gordon, I. R. (2001) · Statistics in Medicine · doi:10.1002/sim.650 · Cited by 573
Bayesian approaches to random-effects meta-analysis: a comparative study
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780142408 · Cited by 568
A LIKELIHOOD APPROACH TO META-ANALYSIS WITH RANDOM EFFECTS
HARDY, R. J., & THOMPSON, S. G. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960330)15:6<619::aid-sim188>3.0.co;2-a · Cited by 522
Restricted Maximum Likelihood (REML) Estimation of Variance Components in the Mixed Model
Corbeil, R. R., & Searle, S. R. (1976) · Technometrics · doi:10.2307/1267913 · Cited by 471
A simple confidence interval for meta-analysis
Sidik, K., & Jonkman, J. N. (2002) · Statistics in Medicine · doi:10.1002/sim.1262 · Cited by 431
A random effects model for effect sizes.
Hedges, L. V. (1983) · Psychological Bulletin · doi:10.1037//0033-2909.93.2.388 · Cited by 399
Evaluation of old and new tests of heterogeneity in epidemiologic meta-analysis
Takkouche, B., Cadarso-Suarez, C., & Spiegelman, D. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009981 · Cited by 380
Overview of randomised trials of diuretics in pregnancy.
Collins, R., Yusuf, S., & Peto, R. (1985) · BMJ · doi:10.1136/bmj.290.6461.17 · Cited by 318
Empirical bayes meta-analysis.
Raudenbush, S. W., & Bryk, A. S. (1985) · Journal of Educational Statistics · doi:10.3102/10769986010002075 · Cited by 299 · cited 1×
“Consequently, a large number of methods have been proposed for estimating the amount of heterogeneity, including two di erent method-of-moments estimators [1, 11], maximumlikelihood [7] and restricted maximum-likelihood estimators [12], and an empirical Bayes estimator [13].”
MethodINCORPORATING VARIABILITY IN ESTIMATES OF HETEROGENEITY IN THE RANDOM EFFECTS MODEL IN META-ANALYSIS
BIGGERSTAFF, B. J., & TWEEDIE, R. L. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970415)16:7<753::aid-sim494>3.0.co;2-g · Cited by 272
A multilevel model framework for meta-analysis of clinical trials with binary outcomes
Turner, R. M., Omar, R. Z., Yang, M., Goldstein, H., & Thompson, S. G. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001230)19:24<3417::aid-sim614>3.0.co;2-l · Cited by 265
Simple Heterogeneity Variance Estimation for Meta-Analysis
Sidik, K., & Jonkman, J. N. (2005) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/j.1467-9876.2005.00489.x · Cited by 253
Investigating causes of heterogeneity in systematic reviews
Glasziou, P. P., & Sanders, S. L. (2002) · Statistics in Medicine · doi:10.1002/sim.1183 · Cited by 213
(2002) · Understanding Social Science Research · doi:10.4135/9780857020208.n7 · Cited by 195
Morrell, C. H. (1998) · Biometrics · doi:10.2307/2533680 · Cited by 168
Hypothesis tests for population heterogeneity in meta‐analysis
Viechtbauer, W. (2006) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711005x64042 · Cited by 145
Self-citationGeneralized linear mixed models for meta-analysis
Platt, R. W., Leroux, B. G., & Breslow, N. E. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990330)18:6<643::aid-sim76>3.0.co;2-m · Cited by 74
On Constructing Confidence Intervals for a Standardized Mean Difference in Meta-analysis
Sidik, K., & Jonkman, J. N. (2003) · Communications in Statistics - Simulation and Computation · doi:10.1081/sac-120023885 · Cited by 57
Tests for Homogeneity of the Risk Difference when Data are Sparse
Lipsitz, S. R., Dear, K. B. G., Laird, N. M., & Molenberghs, G. (1998) · Biometrics · doi:10.2307/2534003 · Cited by 57
Bootstrap estimates of standard errors in validity generalization.
Switzer, F. S., Paese, P. W., & Drasgow, F. (1992) · Journal of Applied Psychology · doi:10.1037/0021-9010.77.2.123 · Cited by 38
Likelihood inference for small variance components
Stern, S., & Welsh, A. H. (2000) · Canadian Journal of Statistics · doi:10.2307/3315962 · Cited by 28
Hartung, J. G., & Knapp, G. (2003) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2003.09.032 · Cited by 23
Explaining heterogeneity in meta-analysis: a comparison of methods
Thompson, S. G., & Sharp, S. J. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991030)18:20
Cited by
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Self-citationA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569
“Some methods allow quantification of uncertainty in the estimate, including a full likelihood approach to the meta-analysis (Hardy and Thompson, 1996), and a method based on a x2-distribution for a random-effects version of the Q-statistic (Viechtbauer, 2007; Knapp et al., 2006); Viechtbauer (2007) reviewed several me…”
MethodMethods to estimate the between‐study variance and its uncertainty in meta‐analysis
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1164 · Cited by 1,685
Self-citationUndue reliance on I 2 in assessing heterogeneity may mislead
Rücker, G., Schwarzer, G., Carpenter, J. R., & Schumacher, M. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-79 · Cited by 1,287
Kane, R. L., Shamliyan, T. A., Mueller, C., Duval, S., & Wilt, T. J. (2007) · Medical Care · doi:10.1097/mlr.0b013e3181468ca3 · Cited by 1,097
Metadata
- Authors
- Viechtbauer, W.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2006-02-06
- Volume
- 26
- Issue
- 1
- Pages
- 37-52
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W2006979162
- Cited by
- 394
- References
- 35
- FWCI
- 10.91
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.