CoChat

Methods for dealing with multiple outcomes in meta-analysis:a comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis

Moeyaert, Mariola, Ugille, Maaike, Natasha Beretvas, S., Ferron, John, Bunuan, Rommel, Van den Noortgate, Wim · 2017 · International Journal of Social Research Methodology

The DOI matches its Crossref record; 41 of the 44 references it cites were found in a registry or an index.

Abstract

This study investigates three methods to handle dependency among effect size estimates in meta-analysis arising from studies reporting multiple outcome measures taken on the same sample. The three-level approach is compared with the method of robust variance estimation, and with averaging effects within studies. A simulation study is performed, and the fixed and random effect estimates of the three methods are compared with each other. Both the robust variance estimation and three-level approach result in unbiased estimates of the fixed effects, corresponding standard errors and variances. Averaging effect sizes results in overestimated standard errors when the effect sizes within studies are truly independent. Although the robust variance and three-level approach are more complicated to use, they have the advantage that they do not require an estimate of the correlation between outcomes, and they still result in unbiased parameter estimates.

Overview

Integrity signals

Registry

Crossref · 10.1080/13645579.2016.1252189

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 41 references

Research works printed but not linked: 4

  1. 10.1002/jrsm.v1:1 · doi:10.1002/jrsm.v1:1
  2. 10.1002/jrsm.v2.1 · doi:10.1002/jrsm.v2.1
  3. SAS Institute Inc, 2011, SAS® 9.3 Software
  4. 10.1002/jrsm.v5.1 · doi:10.1002/jrsm.v5.1

Cited by

See all 275

Metadata

Authors
Moeyaert, M., Ugille, M., Natasha Beretvas, S., Ferron, J., Bunuan, R., & Van den Noortgate, W.
Affiliations
  • State University of New York (US)
  • KU Leuven (BE)
  • Department of Educational Psychology, University of Texas at Austin, Austin, Texas, USA
  • University of South Florida (US)
  • The University of Texas at Austin (US)
  • Department of Psychology and Educational Sciences, University of Leuven, Leuven, Belgium
Corresponding author
Moeyaert, Mariola
Venue
International Journal of Social Research Methodology
Published
2016-11-13
Volume
20
Issue
6
Pages
559-572
ISSN
  • 1364-5579 (ISSN-L)
  • 1464-5300
OpenAlex
W2554874129
Cited by
274
References
44
Citations by year
2026: 30 · 2025: 33 · 2024: 27 · 2023: 36 · 2022: 40 · 2021: 43 · 2020: 26 · 2019: 26 · 2018: 10 · 2017: 3
FWCI
12.25
Type
journal article
Language
en
Open access
closed
Locations
  • International Journal of Social Research Methodology (publishedVersion)
  • Lirias (KU Leuven) (submittedVersion)
Funders
U.S. Department of Education (R305D150007)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Keywords
multiple outcomes, meta-analysis, robust variance estimation, multilevel meta-analysis, three-level model, standard error estimation, fixed and random effects, estimation bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Methods for dealing with multiple outcomes in meta-analysis:a comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis. https://app.cochat.ai/papers/10.1080/13645579.2016.1252189
The paper
Moeyaert, M., Ugille, M., Natasha Beretvas, S., Ferron, J., Bunuan, R., & Van den Noortgate, W. (2017). Methods for dealing with multiple outcomes in meta-analysis:a comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis. International Journal of Social Research Methodology. https://doi.org/10.1080/13645579.2016.1252189

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.