Bias and Efficiency of Meta-Analytic Variance Estimators in the Random-Effects Model
The DOI matches its Crossref record; 39 of the 40 references it cites were found in a registry or an index.
Abstract
The meta-analytic random effects model assumes that the variability in effect size estimates drawn from a set of studies can be decomposed into two parts: heterogeneity due to random population effects and sampling variance. In this context, the usual goal is to estimate the central tendency and the amount of heterogeneity in the population effect sizes. The amount of heterogeneity in a set of effect sizes has implications regarding the interpretation of the meta-analytic findings and often serves as an indicator for the presence of potential moderator variables. Five population heterogeneity estimators were compared in this article analytically and via Monte Carlo simulations with respect to their bias and efficiency.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The second stage pooled the effects across studies by use of random-effects meta-analysis with a restricted maximum likelihood estimator, which is a relatively unbiased and efficient estimator of between-study variance.35”
From Associations between social connections and cognition: a global collaborative individual participant data meta-analysis (2022) · cites this paper as [35]
In the citing paper
“Meta-analyses were conducted using random-effects models with restricted maximum likelihood estimators (Viechtbauer, 2005). Random-effects models were chosen as they compute less biased and more conservative estimated effect sizes compared to fixed-effects models (Card, 2011).”
From The relationship between acceptance of cancer and distress: A meta-analytic review (2019) · cites this paper as Viechtbauer, 2005
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 21×
“…generalization research, which can be attributed to Hunter and Schmidt (1990), an estimator proposed by Hedges (1983, 1989), an estimator by DerSimonian and Laird (1986), and two estimators based on maximum likelihood estimation, the regular maximum likelihood estimator and the restricted…”
BackgroundMethodInfluentialStatistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
A Simplex Method for Function Minimization
Nelder, J. A., & Mead, R. (1965) · The Computer Journal · doi:10.1093/comjnl/7.4.308 · Cited by 28,905 · cited 1×
“In this case, one can employ a direct maximization technique without derivatives, such as the algorithms suggested by Nelder and Mead (1965) and by Byrd, Lu, Nocedal, and Zhu (1995).”
MethodStatistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
A Limited Memory Algorithm for Bound Constrained Optimization
Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995) · SIAM Journal on Scientific Computing · doi:10.1137/0916069 · Cited by 6,221
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 6×
“Modeling of moderator effects is not discussed in the present article, but the interested reader should consult, for example, Hedges (1994), Overton (1998), and Raudenbush (1994). Suffice it to say, any heterogeneity in the population effect sizes, whether caused by moderator effects or random variation, will be refle…”
BackgroundInfluentialMethods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Primary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890 · cited 2×
“Introduction Although the roots of meta-analytic methodology reach back further in time than the conception of the term “meta-analysis” itself (Chalmers, Hedges, & Cooper, 2002; Glass, 1976), most of the research conducted in developing this technique has taken place during the last two decades.”
BackgroundZiegel, E. R., Lehmann, E. L., & Casella, G. (1999) · Technometrics · doi:10.2307/1270597 · Cited by 4,203
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,642
Searle, S. R., & Searle, S. R. (1971) · Cited by 3,229
Fixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems
Harville, D. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10480998 · Cited by 2,667
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings.
Raudenbush, S. W., Hunter, J. E., & Schmidt, F. L. (1991) · Journal of the American Statistical Association · doi:10.2307/2289738 · Cited by 2,327
Searle, S. R., Casella, G., & McCulloch, C. E. (1992) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316856 · Cited by 1,926
(1998) · Springer Texts in Statistics · doi:10.1007/b98854 · Cited by 1,719
Parametric empirical Bayes inference: theory and applications
Morris, C. N. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477920 · Cited by 1,448 · cited 1×
“It appears that Equation 19 originated with Morris (1983), who suggested it as an “approximate” REML estimate (p. 53).”
BackgroundEstimation of a Single Effect Size: Parametric and Nonparametric Methods
Hedges, L. V., & Olkin, I. (1985) · Statistical Methods for Meta-Analysis · doi:10.1016/b978-0-08-057065-5.50010-5 · Cited by 983
A random‐effects regression model for meta‐analysis
Berkey, C. S., Hoaglin, D. C., Mosteller, F., & Colditz, G. A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780140406 · Cited by 879
Hunter, J. E., & Schmidt, F. L. (2000) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00156 · Cited by 833 · cited 1×
“…that the randomand mixed-effects models should be preferred over the decidedly more simple fixed-effects model (Erez, Bloom, & Wells, 1996; Hunter & Schmidt, 2000; National Journal of Educational and Behavioral Statistics Fall 2005, Vol. 30, No. 3, pp. 261–293 The author thanks Howard…”
MethodA brief history of research synthesis
Chalmers, I., Hedges, L. V., & Cooper, H. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001003 · Cited by 552
Restricted Maximum Likelihood (REML) Estimation of Variance Components in the Mixed Model
Corbeil, R. R., & Searle, S. R. (1976) · Technometrics · doi:10.2307/1267913 · Cited by 471
(1994) · Cited by 463
A random effects model for effect sizes.
Hedges, L. V. (1983) · Psychological Bulletin · doi:10.1037//0033-2909.93.2.388 · Cited by 399
Hedges, L. V. (1994) · Cited by 397
Overton, R. C. (1998) · Psychological Methods · doi:10.1037//1082-989x.3.3.354 · Cited by 356
Review of developments in meta-analytic method.
Bangert-Drowns, R. L. (1986) · Psychological Bulletin · doi:10.1037/0033-2909.99.3.388 · Cited by 327
Combining Information: Statistical Issues and Opportunities for Research
Draper, D. L., Gaver, D. P., Goel, P., Greenhouse, J. B., Hedges, L. V., Morris, C., Tucker, J., & Waternaux, C. (1993) · Cited by 279
Combining Information: Statistical Issues and Opportunities for Research
Srinivasan, C. (1993) · Technometrics · doi:10.1080/00401706.1993.10485377 · Cited by 213
EREZ, A., BLOOM, M. C., & WELLS, M. T. (1996) · Personnel Psychology · doi:10.1111/j.1744-6570.1996.tb01801.x · Cited by 161
The explosion of knowledge, references, and citations: Psychology's unique response to a crisis.
Adair, J. G., & Vohra, N. (2003) · American Psychologist · doi:10.1037/0003-066x.58.1.15 · Cited by 90
An unbiased correction for sampling error in validity generalization studies.
Hedges, L. V. (1989) · Journal of Applied Psychology · doi:10.1037//0021-9010.74.3.469 · Cited by 55
The bias of the commonly-used estimate of variance in meta-analysis
Li, Y., Shi, L., & Daniel Roth, H. (1994) · Communications in Statistics - Theory and Methods · doi:10.1080/03610929408831305 · Cited by 45
Estimators of Random Effects Variance Components in Meta-Analysis
Friedman, L. (2000) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165309 · Cited by 23
The Explosion of Knowledge, References, and Citations:
Adair, J. G., VandenBos, G. R., Vohra, N., Arpin, C., Cameron, J. K., Coehlo, A., Derksen, M., Kobylak, J., Macrodimitris, S. D., & Williams, H. (2003) · Cited by 14
Meta-analysis: formulating, evaluating, combining, and reporting
Normand, S.-L. T. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990215)18:3<321::aid-sim28>3.0.co;2-p
Explaining heterogeneity in meta-analysis: a comparison of methods
Thompson, S. G., & Sharp, S. J. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991030)18:20
Research works printed but not linked: 2
- Patterson HD, 1974, Proceedings of the 8th International Biometrics Conference, 197
- Raudenbush SW, 1994, The handbook of research synthesis, 301
Cited by
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
“…function are given by rma.uni(yi, vi, sei, ai, bi, ci, di, n1i, n2i, m1i, m2i, sd1i, sd2i, xi, mi, ri, ni, mods = NULL, data = NULL, intercept = TRUE, slab = NULL, subset = NULL, measure = "GEN", add = 1/2, to = "only0", vtype = "LS", method = "REML", weighted = TRUE, level = 95, digits = 4,…”
MethodInfluentialSelf-citationKowalski, R. M., Giumetti, G. W., Schroeder, A. N., & Lattanner, M. R. (2014) · Psychological Bulletin · doi:10.1037/a0035618 · Cited by 3,266
“…random effects variance component for the mixed-effects model when conducting moderator analyses, we used the restricted maximum likelihood method, as this method of estimation typically provides more accurate estimates than other commonly used approaches (Lipsey & Wilson, 2001; Viechtbauer, 2005).”
MethodOutlier and influence diagnostics for meta-analysis
Viechtbauer, W., & Cheung, M. W.-L. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.11 · Cited by 2,949
Self-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
“Numerous methods are available to estimate the heterogeneity variance in meta-analysis [31].”
MethodAlliance in individual psychotherapy
Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011) · Psychotherapy · doi:10.1037/a0022186 · Cited by 2,072
“All numerical estimates were calculated using restricted maximum likelihood (random-effects) model (Viechtbauer, 2005).”
Method
Metadata
- Authors
- Viechtbauer, W.
- Affiliations
- Maastricht University (NL)
- Corresponding author
- Viechtbauer, Wolfgang
- Venue
- Journal of Educational and Behavioral Statistics
- Published
- 2005-09
- Volume
- 30
- Issue
- 3
- Pages
- 261-293
- ISSN
- 1076-9986 (ISSN-L)
- 1935-1054
- OpenAlex
- W2154449833
- Cited by
- 1,699
- References
- 40
- Citations by year
- 2026: 257 · 2025: 262 · 2024: 238 · 2023: 232 · 2022: 149 · 2021: 99 · 2020: 78 · 2019: 78 · 2018: 63 · 2017: 41 · 2016: 45 · 2015: 32 · 2014: 29 · 2013: 31 · 2012: 14
- FWCI
- 4.20
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Journal of Educational and Behavioral Statistics (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Lirias (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- random effects models, heterogeneity estimation, study heterogeneity, estimation bias, estimator efficiency, sampling variance, moderators
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.