How interchangeable are different estimators of effect size?
Abstract
The computation of effect sizes is a key feature of meta-analysis. In treatment outcome meta-analyses, the standardized mean difference statistic on posttest scores (d) is usually the effect size statistic used. However, when primary studies do not report the statistics needed to compute d, many methods for estimating d from other data have been developed. Little is known about the accuracy of these estimates, yet meta-analysts frequently use them on the assumption that they are estimating the same population parameter as d. This study investigates that assumption empirically. On a sample of 140 psychosocial treatment or prevention studies from a variety of areas, the present study shows that these estimates yield results that are often not equivalent to d in either mean or variance. The frequent mixing of d and other estimates of d in past meta-analyses, therefore, may have led to biased effect size estimates and inaccurate significance tests.
Cites (7)
- Primary, Secondary, and Meta-Analysis of ResearchGLASS, GENE V · 1976 · Educational Researcher
- The earth is round (p < .05).Cohen, Jacob · 1994 · American Psychologist
- Methods of Meta-AnalysisHunter, John, Schmidt, Frank · 2004
- The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis.Lipsey, Mark W., Wilson, David B. · 1993 · American Psychologist
- Gender differences in sexuality: A meta-analysis.Oliver, Mary Beth, Hyde, Janet Shibley · 1993 · Psychological Bulletin
- Random versus nonrandom assignment in controlled experiments: Do you get the same answer?Shadish, William R., Ragsdale, Kevin · 1996 · Journal of Consulting and Clinical Psychology
- Assignment methods in experimentation: When do nonrandomized experiments approximate answers from randomized experiments?Heinsman, Donna T., Shadish, William R. · 1996 · Psychological Methods
Cited by (27)
- Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs.Morris, Scott B., DeShon, Richard P. · 2002 · Psychological Methods
- Is meditation associated with altered brain structure? A systematic review and meta-analysis of morphometric neuroimaging in meditation practitionersFox, Kieran C.R., Nijeboer, Savannah, Dixon, Matthew L., Floman, James L., Ellamil, Melissa, Rumak, Samuel P., Sedlmeier, Peter, Christoff, Kalina · 2014 · Neuroscience & Biobehavioral Reviews
- Social influence approaches to encourage resource conservation: a meta-analysisAbrahamse, Wokje, Steg, Linda · 2013 · Global Environmental Change
- Leadership training design, delivery, and implementation: A meta-analysis.Lacerenza, Christina N., Reyes, Denise L., Marlow, Shannon L., Joseph, Dana L., Salas, Eduardo · 2017 · Journal of Applied Psychology
- Paradoxical effects of thought suppression: a meta-analysis of controlled studiesAbramowitz, Jonathan S, Tolin, David F, Street, Gordon P · 2001 · Clinical Psychology Review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How interchangeable are different estimators of effect size? https://app.cochat.ai/papers/10.1037//0022-006x.64.6.1316
- The paper
- Ray, J. W., & Shadish, W. R. (1996). How interchangeable are different estimators of effect size? Journal of Consulting and Clinical Psychology. https://doi.org/10.1037//0022-006x.64.6.1316
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.