Hierarchical Dependence in Meta-Analysis
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Meta-analysis is a frequent tool among education and behavioral researchers to combine results from multiple experiments to arrive at a clear understanding of some effect of interest. One of the traditional assumptions in a meta-analysis is the independence of the effect sizes from the studies under consideration. This article presents a meta-analytic review of 13 experiments with 18 study reports all involving the effect of native-language (L1) vocabulary aids on second-language (L2) reading comprehension. Some experiments produced multiple study reports, creating a dependence structure among the resulting effect size estimates. The covariance among these effect size estimates is estimated and incorporated into a proposed meta-analysis model that accounts for the dependence at a hierarchical level. The overall effect size estimate (g =. 63) indicates that L1 vocabulary aids can be an effective L2 reading comprehension aid in the short term. An interpretation of the hierarchical components is discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 4×
“as in chapter 5 of Hedges and Olkin (1985). Note that if the study used a single-factor ANOVA model to compare two groups (e.”
BackgroundMethodInfluentialPrimary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890
Estimation of a Single Effect Size: Parametric and Nonparametric Methods
Hedges, L. V., & Olkin, I. (1985) · Statistical Methods for Meta-Analysis · doi:10.1016/b978-0-08-057065-5.50010-5 · Cited by 983
HULSTIJN, J. H., HOLLANDER, M., & GREIDANUS, T. (1996) · The Modern Language Journal · doi:10.1111/j.1540-4781.1996.tb01614.x · Cited by 760
Jacobs, G. M., Dufon, P., & Hong, F. C. (1994) · Journal of Research in Reading · doi:10.1111/j.1467-9817.1994.tb00049.x · Cited by 247
A multivariate mixed linear model for meta-analysis.
Kalaian, H. A., & Raudenbush, S. W. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.3.227 · Cited by 229 · cited 2×
“Kalaian and Raudenbush (1996); Berkey, Hoaglin, AntczakBouckoms, Mosteller, and Colditz (1998); and Nam, Mengersen, and Garthwaite (2003) discuss methods to account for multivariate meta-analysis situations where individual studies report estimates for different (but possibly correlated) effect…”
BackgroundNam, I., Mengersen, K., & Garthwaite, P. (2003) · Statistics in Medicine · doi:10.1002/sim.1410 · Cited by 153 · cited 1×
“Kalaian and Raudenbush (1996); Berkey, Hoaglin, AntczakBouckoms, Mosteller, and Colditz (1998); and Nam, Mengersen, and Garthwaite (2003) discuss methods to account for multivariate meta-analysis situations where individual studies report estimates for different (but possibly correlated) effect…”
BackgroundPictures and Second Language Comprehension: Do They Help?
Omaggio, A. C. (1979) · Foreign Language Annals · doi:10.1111/j.1944-9720.1979.tb00153.x · Cited by 146
Cited by
Robust variance estimation in meta‐regression with dependent effect size estimates
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.5 · Cited by 2,086
“Methods for meta-analysis in such situations have been discussed by Konstantopoulos [20] and Stevens and Taylor [21].”
MethodCheung, M. W.-L. (2014) · Psychological Methods · doi:10.1037/a0032968 · Cited by 1,342
Three-level meta-analysis of dependent effect sizes
Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0261-6 · Cited by 1,130
“Yet the dependence over studies and, especially, the dependence within studies including independent samples are often overlooked, although recently, Stevens and Taylor (2009) and Hedges, Tipton and Johnson (2010) discussed the use of random effects for dealing with this kind of dependence.”
BackgroundMethodPlant–soil feedbacks: a meta‐analytical review
Kulmatiski, A., Beard, K. H., Stevens, J. R., & Cobbold, S. M. (2008) · Ecology Letters · doi:10.1111/j.1461-0248.2008.01209.x · Cited by 983
Self-citationLand‐use intensity and the effects of organic farming on biodiversity: a hierarchical meta‐analysis
Tuck, S. L., Winqvist, C., Mota, F., Ahnström, J., Turnbull, L. A., & Bengtsson, J. (2014) · Journal of Applied Ecology · doi:10.1111/1365-2664.12219 · Cited by 846
Metadata
- Authors
- Stevens, J. R., & Taylor, A. M.
- Venue
- Journal of Educational and Behavioral Statistics
- Published
- 2009-03
- Volume
- 34
- Issue
- 1
- Pages
- 46-73
- ISSN
- 1076-9986 (ISSN-L)
- 1935-1054
- OpenAlex
- W1970886815
- Cited by
- 93
- References
- 8
- FWCI
- 4.12
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.