How interchangeable are different estimators of effect size?

Ray, JoAnne W., Shadish, William R.·1996·Journal of Consulting and Clinical Psychology

Abstract

The computation of effect sizes is a key feature of meta-analysis. In treatment outcome meta-analyses, the standardized mean difference statistic on posttest scores (d) is usually the effect size statistic used. However, when primary studies do not report the statistics needed to compute d, many methods for estimating d from other data have been developed. Little is known about the accuracy of these estimates, yet meta-analysts frequently use them on the assumption that they are estimating the same population parameter as d. This study investigates that assumption empirically. On a sample of 140 psychosocial treatment or prevention studies from a variety of areas, the present study shows that these estimates yield results that are often not equivalent to d in either mean or variance. The frequent mixing of d and other estimates of d in past meta-analyses, therefore, may have led to biased effect size estimates and inaccurate significance tests.

Cites (7)

  1. Primary, Secondary, and Meta-Analysis of Research
    GLASS, GENE V · 1976 · Educational Researcher
  2. The earth is round (p < .05).
    Cohen, Jacob · 1994 · American Psychologist
  3. Methods of Meta-Analysis
    Hunter, John, Schmidt, Frank · 2004
  4. The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis.
    Lipsey, Mark W., Wilson, David B. · 1993 · American Psychologist
  5. Gender differences in sexuality: A meta-analysis.
    Oliver, Mary Beth, Hyde, Janet Shibley · 1993 · Psychological Bulletin
  6. Random versus nonrandom assignment in controlled experiments: Do you get the same answer?
    Shadish, William R., Ragsdale, Kevin · 1996 · Journal of Consulting and Clinical Psychology
  7. Assignment methods in experimentation: When do nonrandomized experiments approximate answers from randomized experiments?
    Heinsman, Donna T., Shadish, William R. · 1996 · Psychological Methods

Show all 7 references

Cited by (27)

  1. Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs.
    Morris, Scott B., DeShon, Richard P. · 2002 · Psychological Methods
  2. Is meditation associated with altered brain structure? A systematic review and meta-analysis of morphometric neuroimaging in meditation practitioners
    Fox, Kieran C.R., Nijeboer, Savannah, Dixon, Matthew L., Floman, James L., Ellamil, Melissa, Rumak, Samuel P., Sedlmeier, Peter, Christoff, Kalina · 2014 · Neuroscience & Biobehavioral Reviews
  3. Social influence approaches to encourage resource conservation: a meta-analysis
    Abrahamse, Wokje, Steg, Linda · 2013 · Global Environmental Change
  4. Leadership training design, delivery, and implementation: A meta-analysis.
    Lacerenza, Christina N., Reyes, Denise L., Marlow, Shannon L., Joseph, Dana L., Salas, Eduardo · 2017 · Journal of Applied Psychology
  5. Paradoxical effects of thought suppression: a meta-analysis of controlled studies
    Abramowitz, Jonathan S, Tolin, David F, Street, Gordon P · 2001 · Clinical Psychology Review

See all 137

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: How interchangeable are different estimators of effect size? https://app.cochat.ai/papers/10.1037//0022-006x.64.6.1316
The paper
Ray, J. W., & Shadish, W. R. (1996). How interchangeable are different estimators of effect size? Journal of Consulting and Clinical Psychology. https://doi.org/10.1037//0022-006x.64.6.1316

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.