Averaging Dependent Effect Sizes in Meta-Analysis: a Cautionary Note about Procedures
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
When a primary study includes several indicators of the same construct, the usual strategy to meta-analytically integrate the multiple effect sizes is to average them within the study. In this paper, the numerical and conceptual differences among three procedures for averaging dependent effect sizes are shown. The procedures are the simple arithmetic mean, the Hedges and Olkin (1985) procedure, and the Rosenthal and Rubin (1986) procedure. Whereas the simple arithmetic mean ignores the dependence among effect sizes, both the procedures by Hedges and Olkin and Rosenthal and Rubin take into account the correlational structure of the effect sizes, although in a different way. Rosenthal and Rubin's procedure provides the effect size for a single composite variable made up of the multiple effect sizes, whereas Hedges and Olkin's procedure presents an effect size estimate of the standard variable. The three procedures were applied to 54 conditions, where the magnitude and homogeneity of both effect sizes and correlation matrix among effect sizes were manipulated. Rosenthal and Rubin's procedure showed the highest estimates, followed by the simple mean, and the Hedges and Olkin procedure, this last having the lowest estimates. These differences are not trivial in a meta-analysis, where the aims must guide the selection of one of the procedures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Writing meta-analytic reviews.
Rosenthal, R. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.183 · Cited by 1,108 · cited 2×
“E-mail: fulmarin@fcu.um.es 32 Averaging Dependent Effect Sizes in Meta-Analysis: a Cautionary Note about Procedures Fulgencio Marín-Martínez and Julio Sánchez-Meca University of Murcia Meta-analysis has recently become a widely used methodology in the social and behavioral sciences.”
BackgroundMethodA multivariate mixed linear model for meta-analysis.
Kalaian, H. A., & Raudenbush, S. W. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.3.227 · Cited by 229 · cited 2×
“In fact, there is not any work dealing with these differences and the existing works referring to both procedures present them as similar (e.g., Abrami, Cohen, & d’Apollonia, 1988; Johnson & Eagly, in press; Kalaian & Raudenbush, 1996; Matt & Cook, 1994; Rosenthal, 1995).”
BackgroundMethod
Research works printed but not linked: 21
- Gleser, 1994, The handbook of research synthesis, 339
- 10.1111/j.2044-8317.1998.tb00683.x · DOI · Paper page
- Johnson, The handbook of research methods in social psychology
- 10.1037/0033-2909.105.1.106 · doi:10.1037/0033-2909.105.1.106 · Paper page
- Sánchez-Meca, 1989, Tratado de psicología general. I: Historia, teoría y método, 617
- Cohen, 1988, Statistical power analysis for the behavioral sciences
- 10.1177/0013164498058002005 · DOI · Paper page
- Glass, 1981, Meta-analysis in social research
- 10.1037/0033-2909.99.3.400 · doi:10.1037/0033-2909.99.3.400 · Paper page
- 1992, The GAUSS System (Vers. 3.0)
- 10.4135/9781412984997 · DOI · Paper page
- 10.3102/00346543058002151 · doi:10.3102/00346543058002151 · Paper page
- Cooper, 1989, Integrating research: A guide for literature reviews
- Hedges, 1985, Statistical methods for meta-analysis
- Hunter, 1990, Methods of meta-analysis: Correcting error and bias in research findings
- 10.1080/00220973.1988.10806497 · DOI
- 10.1080/00220979809598345 · DOI · Paper page
- Matt, 1994, The handbook of research synthesis, 503
- 10.1037/0033-2909.103.1.111 · DOI · Paper page
- Rosenthal, 1994, The handbook of research synthesis, 231
- 10.1023/A:1004298118485 · DOI
Cited by
Unemployment impairs mental health: Meta-analyses
Paul, K. I., & Moser, K. (2009) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2009.01.001 · Cited by 2,706
Cheung, M. W.-L. (2014) · Psychological Methods · doi:10.1037/a0032968 · Cited by 1,342
Sanders, M. R., Kirby, J. N., Tellegen, C. L., & Day, J. J. (2014) · Clinical Psychology Review · doi:10.1016/j.cpr.2014.04.003 · Cited by 1,077
A Meta-Analysis of Compassion-Based Interventions: Current State of Knowledge and Future Directions
Kirby, J. N., Tellegen, C. L., & Steindl, S. R. (2017) · Behavior Therapy · doi:10.1016/j.beth.2017.06.003 · Cited by 827
Working memory and second language comprehension and production: A meta-analysis
Linck, J. A., Osthus, P., Koeth, J. T., & Bunting, M. F. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0565-2 · Cited by 628
Metadata
- Authors
- Marín-Martínez, F., & Sánchez-Meca, J.
- Venue
- The Spanish Journal of Psychology
- Publisher
- Cambridge University Press (CUP)
- Published
- 1999-05
- Volume
- 2
- Issue
- 1
- Pages
- 32-38
- ISSN
- 1138-7416 (ISSN-L)
- 1988-2904
- OpenAlex
- W2046859032
- Cited by
- 99
- References
- 2
- FWCI
- 1.79
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Averaging Dependent Effect Sizes in Meta-Analysis: a Cautionary Note about Procedures. https://app.cochat.ai/papers/10.1017/s1138741600005436
- The paper
- Marín-Martínez, F., & Sánchez-Meca, J. (1999). Averaging Dependent Effect Sizes in Meta-Analysis: a Cautionary Note about Procedures. The Spanish Journal of Psychology. https://doi.org/10.1017/s1138741600005436
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.