Commentary on “A review of effect sizes and their confidence intervals, Part I: The Cohen’s d family”: The degrees of freedom for paired samples designs
Open accessAbstract
In their review of effect sizes of the Cohen's d family, Goulet-Pelletier and Cousineau (2018) proposed several methods for generating confidence intervals for the unbiased standardized mean difference, g.Among them they proposed using degrees of freedom ν = 2(n -1) instead of ν = (n -1) for all paired samples designs that use a pooled standard deviation to standardize the mean difference (pooled paired samples) when calculating g and its confidence limits from a noncentral t distribution.Simulations demonstrate that the exact ν for a pooled paired samples design vary as a function of the population correlation ρ between 2(n -1) at ρ = .0and (n -1) at ρ = 1.0.This affects the calculation of g and the selection of the appropriate noncentral t distribution for calculating the confidence limits.Using a sample r to estimate the unknown ρ causes a further deviation from the presumed noncentral t distribution even when the ν are known.These facts adversely affect the coverage of the confidence intervals computed as recommended by the authors.These methods for calculating noncentral t confidence intervals should not be used as described with pooled paired samples designs.Noncentral t confidence intervals for either a two sample design or a paired samples design where the mean difference is standardized by the standard deviation of the difference scores are unaffected by this problem.An R script and C source code are provided.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Effect sizes computed for repeated measures designs and independent groups designs are not directly comparable because different degrees of freedom are used to calculate each.
“Effect sizes computed for repeated measures designs and independent groups designs are not directly comparable because different degrees of freedom are used to calculate each [62,63].”
From Menstrual cycle effects on cognitive performance: A meta-analysis (2025) · cites this paper as [62]
The Discussion explicitly says that a pooled-standard-deviation effect size (dP) from paired measurements can be compared with an effect size from two independent groups. It says only that the difference-score effect size (dD) cannot be compared directly with the independent-groups dP; the claim incorrectly applies that limitation to repeated-measures effect sizes generally and attributes it to different degrees of freedom. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,385 · see citing works
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225 · see citing works
Lakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303 · see citing works
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · see citing works
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,832 · see citing works
(2011) · Cited by 2,025 · see citing works
Meta-analysis of experiments with matched groups or repeated measures designs
Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.170 · Cited by 1,693 · see citing works
Estimation of effect size from a series of independent experiments.
Hedges, L. V. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.92.2.490 · Cited by 1,065 · see citing works
Cumming, G., & Finch, S. (2001) · Educational and Psychological Measurement · doi:10.1177/0013164401614002 · Cited by 528 · see citing works
Unbiased Estimation of Certain Correlation Coefficients
Olkin, I., & Pratt, J. W. (1958) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177706717 · Cited by 491 · see citing works
A review of effect sizes and their confidence intervals, Part I: The Cohen's d family
Goulet‐Pelletier, J., & Cousineau, D. (2018) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.14.4.p242 · Cited by 419 · see citing works
Confidence Intervals for Standardized Effect Sizes: Theory, Application, and Implementation
Kelley, K. (2007) · Journal of Statistical Software · doi:10.18637/jss.v020.i08 · Cited by 237 · see citing works
Distribution of the standardized mean change effect size for meta‐analysis on repeated measures
Morris, S. B. (2000) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711000159150 · Cited by 206 · see citing works
Bias in Estimation and Hypothesis Testing of Correlation
Zimmerman, D. W., Zumbo, B. D., Williams, R. H., & Zimmerman, D. W. (2003) · Cited by 107 · see citing works
Algorithm AS 243: Cumulative Distribution Function of the Non-Central t Distribution
Lenth, R. V. (1989) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347693 · Cited by 51 · see citing works
Goulet‐Pelletier, J., & Cousineau, D. (2020) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.16.4.p422 · Cited by 15 · see citing works
Goulet‐Pelletier, J., & Cousineau, D. (2019) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.15.1.p054 · Cited by 13 · see citing works
Fitts, D. A. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0968-5 · Cited by 7 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Menstrual cycle effects on cognitive performance: A meta-analysis
Jang, D., Zhang, J., & Elfenbein, H. A. (2025) · PLOS ONE · doi:10.1371/journal.pone.0318576 · Cited by 21 · see citing works
Schmidt, M., Schoenenberg, K., Engelkamp, J. E., Staufenbiel, T., Martin, A., Ebert, D. D., & Hartmann, A. S. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-06797-1 · Cited by 8 · see citing works
- Authors
- Fitts, D. A.
- Affiliations
- University of Washington (US)
- Corresponding author
- Fitts, Douglas A.
- ORCID
- Fitts, Douglas A.
- Venue
- The Quantitative Methods for Psychology
- Published
- 2020-05-01
- Volume
- 16
- Issue
- 4
- Pages
- 281-294
- ISSN
- 2292-1354 (ISSN-L)
- OpenAlex
- W4254475121
- Cited by
- 17
- References
- 18
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 2 · 2023: 3 · 2022: 2 · 2021: 1 · 2020: 1
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- The Quantitative Methods for Psychology (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- Cohen's d, degrees of freedom, noncentral t distribution, pooled standard deviation, population correlation, confidence interval coverage, standardized mean difference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
