Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach
The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Abstract
Meta-analysis is an indispensable tool used to synthesize research findings in the social, educational, medical, management, and behavioral sciences. Most meta-analytic models assume independence among effect sizes. However, effect sizes can be dependent for various reasons. For example, studies might report multiple effect sizes on the same construct, and effect sizes reported by participants from the same cultural group are likely to be more similar than those reported by other cultural groups. This article reviews the problems and common methods to handle dependent effect sizes. The objective of this article is to demonstrate how 3-level meta-analyses can be used to model dependent effect sizes. The advantages of the structural equation modeling approach over the multilevel approach with regard to conducting a 3-level meta-analysis are discussed. This article also seeks to extend the key concepts of Q statistics, I2, and R2 from 2-level meta-analyses to 3-level meta-analyses. The proposed procedures are implemented using the open source metaSEM package for the R statistical environment. Two real data sets are used to illustrate these procedures. New research directions related to 3-level meta-analyses are discussed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Therefore, this study employed the three-level meta-analysis model proposed by Cheung (2014), which decomposes variance into three levels: Level 1, sampling variance; Level 2, within-study variance across multiple effect sizes from the same study; and Level 3, between-study variance across independent studies.”
From Mindfulness- and acceptance-based interventions for music performance anxiety: a three-level meta-analysis of therapeutic effects, mechanisms, and boundary conditions (2026) · cites this paper as Cheung, 2014
In the citing paper
“A major advantage of this three-level approach to meta-analysis is that all relevant effects reported in each primary study can be included, implying that all relevant information is preserved.”
From Risk Factors for School Absenteeism and Dropout: A Meta-Analytic Review (2019) · cites this paper as Cheung 2014
In the citing paper
“As we included five studies that provided two effect sizes, this violates the normal requirement of independent effect size measures in meta-analysis (Cheung, 2014).”
From Therapeutic games to reduce anxiety and depression in young people: A systematic review and exploratory meta‐analysis of their use and effectiveness. (2024) · cites this paper as Cheung, 2014
In the citing paper
“To account for the dependency of estimates at multiple times within studies, we fit three-level meta-analytic models with time periods (short-term or long-term) as a moderator.26”
From Adherence to PCSK9 Inhibitors in Clinical Practice (2026) · cites this paper as [26]
In the citing paper
“We thus adopted a three-level meta-analytic model in which effect sizes are nested within experiments, so we explicitly took the dependencies in our data into account in the model63,64.”
From When combinations of humans and AI are useful: A systematic review and meta-analysis (2024) · cites this paper as [64]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology · doi:10.2307/2075823 · Cited by 18,724
mice: Multivariate Imputation by Chained Equations in R
Buuren, S. V., & Groothuis-Oudshoorn, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v045.i03 · Cited by 13,870 · cited 1×
“Second, conducting MI for multilevel data is challenging (cf. van Buuren & Groothuis-Oudshoorn, 2011).”
MethodMultilevel analysis : an introduction to basic and advanced multilevel modeling
Snijders, T. A. B., & Bosker, R. (1999) · Cited by 13,775
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Mixed-Effects Models in S and S-PLUS
(2001) · Technometrics · doi:10.1198/tech.2001.s574 · Cited by 9,278
(2010) · Cited by 7,034
A basic introduction to fixed-effect and random-effects models for meta-analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.12 · Cited by 6,793 · cited 1×
“On the other hand, random-effects models allow for unconditional inferences to be generalized beyond the studies included under the assumption that the studies were randomly selected (see Borenstein et al., 2010; Hedges & Vevea, 1998).”
MethodMethods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Cited by
Fitting three-level meta-analytic models in R: A step-by-step tutorial
Assink, M., & Wibbelink, C. J. M. (2016) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.12.3.p154 · Cited by 1,330
The relation between short-term emotion dynamics and psychological well-being: A meta-analysis.
Houben, M., Van Den Noortgate, W., & Kuppens, P. (2015) · Psychological Bulletin · doi:10.1037/a0038822 · Cited by 1,043
Sexual assault victimization and psychopathology: A review and meta-analysis
Dworkin, E. R., Menon, S. V., Bystrynski, J., & Allen, N. E. (2017) · Clinical Psychology Review · doi:10.1016/j.cpr.2017.06.002 · Cited by 825
Mertens, S., Herberz, M., Hahnel, U. J. J., & Brosch, T. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2107346118 · Cited by 738
metaSEM: an R package for meta-analysis using structural equation modeling
Cheung, M. W.-L. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01521 · Cited by 685
Self-citation
Metadata
- Authors
- Cheung, M. W.-L.
- Affiliations
- National University of Singapore (SG)
- Corresponding author
- Cheung, Mike W.-L.
- ORCID
- Cheung, Mike W.-L.
- Venue
- Psychological Methods
- Published
- 2014
- Volume
- 19
- Issue
- 2
- Pages
- 211-229
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2122962297
- Semantic Scholar
- 80381b0d516d07e42e308f983624bffd497b1d9b
- Cited by
- 1,342
- References
- 73
- Citations by year
- 2026: 294 · 2025: 203 · 2024: 174 · 2023: 157 · 2022: 143 · 2021: 91 · 2020: 80 · 2019: 69 · 2018: 41 · 2017: 29 · 2016: 27 · 2015: 22 · 2014: 8
- FWCI
- 22.07
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Ministry of Education (R581-000-111-112)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- dependent effect sizes, structural equation modeling, multilevel meta-analysis, q-statistics, I2 statistic, cultural similarity, heterogeneity measures
- MeSH terms
- Humans, Models, Psychological, Research Design, Meta-Analysis as Topic, Models, Statistical, Multivariate Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach. https://app.cochat.ai/papers/10.1037/a0032968
- The paper
- Cheung, M. W.-L. (2014). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach. Psychological Methods. https://doi.org/10.1037/a0032968
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.