New approaches for estimation of effect sizes and their confidence intervals for treatment effects from randomized controlled trials
Open accessAbstract
Although Cohen's d and the growth modeling analysis (GMA) d from linear models are common standardized effect sizes used to convey treatment effects, popular statistical software packages do not include them in their standard outputs.This article demonstrated the use of statistical software with user-prescribed parameter functions (e.g., Mplus) to produce d for treatment effects from both classical analysis and GMA-along with their associated standard errors (SEs) and confidence intervals (CIs).A Monte Carlo study was conducted to examine bias in the SE and CI for GMA d obtained with Mplus and found that both estimates were more accurate when calculated by the software with the standard bootstrap than with the delta method, but the delta method estimates were less biased than respective estimates from extant post hoc equations.Thus, users of many statistical software packages (including SAS, R, and LISREL) should obtain d or GMA d and associated CIs directly.Researchers employing less versatile software-and meta-analysts including ds and GMA ds in their syntheses of treatment effects-should continue to use the conventional post hoc equations.Biases in SEs and CIs for effect sizes obtained with them are ignorable and point estimates of d and GMA d are the same whether obtained directly from the software or with post hoc equations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The effect size calculated from observed values was Cohen's d = 0.96, while the model-predicted effect size based on the multilevel model was dGMA-raw = 0.86.
“The effect size calculated from observed values was Cohen's d = 0.96, while the model‐predicted effect size based on the multilevel model was d GMA‐raw = 0.86, (Feingold 2019).”
From Internet‐Based Acceptance and Commitment Therapy With Interoceptive Exposure for Panic Disorder: A Randomized Controlled Trial and Working Alliance Analysis. (2026) · cites this paper as Feingold 2019
The sections “Calculation of d in Classical Analysis” and “Calculating GMA d from Extant Post Hoc Equations” distinguish Cohen’s d based on observed group means from a model-estimated GMA d. But the paper does not report the claimed pair of values, 0.96 and 0.86; its illustrative multiple-covariate analysis reports GMA d = 0.872. Page numbers were not available in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(2013) · Cited by 44,563 · see citing works
<b>lavaan</b> : An <i>R</i> Package for Structural Equation Modeling
Rosseel, Y. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i02 · Cited by 27,955 · see citing works
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230 · see citing works
Mediation in experimental and nonexperimental studies: New procedures and recommendations.
Shrout, P. E., & Bolger, N. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.4.422 · Cited by 11,335 · see citing works
Confidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods
MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3901_4 · Cited by 7,806 · see citing works
Multilevel analysis techniques and applications
Hox, J. J., Moerbeek, M., & van de Schoot, R. (2002) · Cited by 6,005 · see citing works
Introduction to Statistical Mediation Analysis
MacKinnon, D. P. (2012) · doi:10.4324/9780203809556 · Cited by 5,618 · see citing works
Preacher, K. J., & Kelley, K. (2011) · Psychological Methods · doi:10.1037/a0022658 · Cited by 3,060 · see citing works
Morris, S. B., & DeShon, R. P. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.105 · Cited by 2,502 · see citing works
How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power
Muthén, L. K., & Muthén, B. O. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0904_8 · Cited by 2,362 · see citing works
Gender differences in personality: A meta-analysis.
Feingold, A. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.116.3.429 · Cited by 2,240 · see citing works
Self-citationEstimating effect sizes from pretest-posttest-control group designs
Morris, S. B. (2008) · Organizational Research Methods · doi:10.1177/1094428106291059 · Cited by 2,133 · see citing works
Bootstrapping goodness-of-fit measures in structural equation models.
Bollen, K. A., & Stine, R. A. (1992) · Sociological Methods & Research · doi:10.1177/0049124192021002004 · Cited by 1,831 · see citing works
Early vocabulary growth: relation to language input and gender
Huttenlocher, J., Haight, W., Bryk, A., Seltzer, M., & Lyons, T. (1991) · Developmental Psychology · doi:10.1037/0012-1649.27.2.236 · Cited by 1,783 · see citing works
Gueorguieva, R., & Krystal, J. H. (2004) · Archives of General Psychiatry · doi:10.1001/archpsyc.61.3.310 · Cited by 1,529 · see citing works
Latent Curve Models: A Structural Equation Perspective
Horan, P. M. (2006) · Contemporary Sociology A Journal of Reviews · doi:10.1177/009430610603500650 · Cited by 1,214 · see citing works
Kelley, K., & Preacher, K. J. (2012) · Psychological Methods · doi:10.1037/a0028086 · Cited by 1,002 · see citing works
Feingold, A. (2009) · Psychological Methods · doi:10.1037/a0014699 · Cited by 851 · see citing works
Self-citationUnderstanding The New Statistics
Cumming, G. (2013) · Routledge eBooks · doi:10.4324/9780203807002 · Cited by 839 · see citing works
Donald, H., & Robert D., G. (2006) · Wiley Series in Probability and Statistics · doi:10.1002/0470036486 · Cited by 821 · see citing works
Standardized or simple effect size: what should be reported?
Baguley, T. (2009) · British Journal of Psychology · doi:10.1348/000712608x377117 · Cited by 619 · see citing works
Grissom, R. J., & Kim, J. J. (2012) · doi:10.4324/9780203803233 · Cited by 470 · see citing works
An Introduction to Meta–Analysis
Konstantopoulos, S. (2008) · Sage eBooks · doi:10.4135/9781412995627.d15 · Cited by 434 · see citing works
A review of effect sizes and their confidence intervals, Part I: The Cohen's d family
Goulet‐Pelletier, J., & Cousineau, D. (2018) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.14.4.p242 · Cited by 419 · see citing works
Estimating and Comparing Specific Mediation Effects in Complex Latent Variable Models
Lau, R. S., & Cheung, G. W. (2010) · Organizational Research Methods · doi:10.1177/1094428110391673 · Cited by 373 · see citing works
A standardized mean difference effect size for single case designs
Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1052 · Cited by 308 · see citing works
Comparison of methods for constructing confidence intervals of standardized indirect effects
Cheung, M. W. ‐. (2009) · Behavior Research Methods · doi:10.3758/brm.41.2.425 · Cited by 279 · see citing works
Chorpita, B. F., Daleiden, E. L., Park, A. L., Ward, A. M., Levy, M. C., Cromley, T., Chiu, A. W., Letamendi, A. M., Tsai, K. H., & Krull, J. L. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000133 · Cited by 210 · see citing works
A Regression Framework for Effect Size Assessments in Longitudinal Modeling of Group Differences
Feingold, A. (2012) · Review of General Psychology · doi:10.1037/a0030048 · Cited by 199 · see citing works
Self-citationGender Differences in Mathematical Trajectories
Leahey, E., & Guo, G. (2001) · Social Forces · doi:10.1353/sof.2001.0102 · Cited by 166 · see citing works
Feingold, A. (2014) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0037721 · Cited by 126 · see citing works
Self-citationParra‐Cardona, J. R., Bybee, D. I., Sullivan, C. M., Rodríguez, M. M. D., Dates, B. G., Tams, L., & Bernal, G. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000160 · Cited by 111 · see citing works
Stice, E., Rohde, P., Shaw, H., & Gau, J. M. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000211 · Cited by 103 · see citing works
Confidence Intervals for Effect Sizes: Applying Bootstrap Resampling
Banjanovic, E. S., & Osborne, J. W. (2020) · Scholarworks (University of Massachusetts Amherst) · doi:10.7275/dz3r-8n08 · Cited by 100 · see citing works
A Delta Method for Implicitly Defined Random Variables
Bénichou, J., & Gail, M. H. (1989) · The American Statistician · doi:10.1080/00031305.1989.10475608 · Cited by 85 · see citing works
Tutorial on Using Regression Models with Count Outcomes using R
Beaujean, A. A., & Morgan, G. B. (2020) · Scholarworks (University of Massachusetts Amherst) · doi:10.7275/pj8c-h254 · Cited by 64 · see citing works
Odgaard, E. C., & Fowler, R. L. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019294 · Cited by 60 · see citing works
Felder, J. N., Epel, E. S., Lewis, J. B., Cunningham, S. D., Tobin, J. N., Rising, S. S., Thomas, M., & Ickovics, J. R. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000191 · Cited by 56 · see citing works
Feingold, A., Mackinnon, D., & Capaldi, D. (2018) · Addictive Behaviours · doi:10.1016/j.addbeh.2018.12.018 · Cited by 36 · see citing works
Self-citationA cautionary note on modeling growth trends in longitudinal data.
Kuljanin, G., Braun, M. T., & DeShon, R. P. (2011) · Psychological Methods · doi:10.1037/a0023348 · Cited by 34 · see citing works
Time-Varying Effect Sizes for Quadratic Growth Models in Multilevel and Latent Growth Modeling
Feingold, A. (2018) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2018.1547110 · Cited by 29 · see citing works
Self-citationMeta-analysis with standardized effect sizes from multilevel and latent growth models.
Feingold, A. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000162 · Cited by 26 · see citing works
Self-citationGoodnight, J. A., Bates, J. E., Holtzworth‐Munroe, A., Pettit, G. S., Ballard, R. H., Iskander, J. M., Swanson, A., Dodge, K. A., & Lansford, J. E. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000226 · Cited by 26 · see citing works
Brick, L. A. D., & Harlow, L. L. (2023) · Routledge eBooks · doi:10.4324/9780429269103-19 · Cited by 2 · see citing works
Flagged first, then by citations
1 of the 45 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Cohen J, Cohen P, West SG, & Aiken LS (2003). Applied multiple regression/correlation analysis for the behavioral analysis (3rd ed.). Mahwah, NJ: Erlbaum.
- Goldstein H (2011). Multilevel statistical models (4th ed.). Hobokin, NJ: Wiley.
- Kendall M, & Stuart A (1977). The advanced theory of statistics: Volume 1 (4th edition). New York: MacMillan.
- Preacher KJ, & Kelley K (2011). Effect size measures for mediation models: Quantitative strategies for communicating indirect effects. Psychological Methods, 16, 93– 115. 21500915
Cited by
Sorted by citations
Beauchamp, M. R., Hulteen, R. M., Ruissen, G. R., Liu, Y., Rhodes, R. E., Wierts, C. M., Waldhauser, K. J., Harden, S. M., & Puterman, E. M. (2021) · Journal of Medical Internet Research · doi:10.2196/30709 · Cited by 70 · see citing works
Boucher, V. G., Haight, B. L., Hives, B. A., Zumbo, B. D., Merali-Dewji, A., Hutton, S., Liu, Y., Nguyen, S., Beauchamp, M. R., Black, A., et al. (2023) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2023.2706 · Cited by 39 · see citing works
Wu, X., Zhang, K., Kuang, N., Kong, X., Cao, M., Lian, Z., Liu, Y., Fan, H., Yu, G., Liu, Z., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-61785-z · Cited by 23 · see citing works
Puterman, E. M., Hives, B. A., Mazara, N., Grishin, N. K., Webster, J., Hutton, S., Koehle, M. S., Liu, Y., & Beauchamp, M. R. (2021) · British Journal of Sports Medicine · doi:10.1136/bjsports-2021-104379 · Cited by 22 · see citing works
Vranceanu, A., Manglani, H. R., Choukas, N. R., Kanaya, M. R., Lester, E. G., Zale, E. L., Plotkin, S. R., Jordan, J. T., Macklin, E. A., & Bakhshaie, J. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.20599 · Cited by 17 · see citing works
Effects of Youth Flexible Assertive Community Treatment: outcomes of an 18-month observational study
Broersen, M., Creemers, D. H. M., Frieswijk, N., Vermulst, A. A., & Kroon, H. (2023) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-023-02508-x · Cited by 15 · see citing works
Lee, K. S., Xiao, J., Luo, J., Leibenluft, E., Liew, Z., & Tseng, W. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.803891 · Cited by 14 · see citing works
Online Transdiagnostic Emotion Regulation Treatment for Adolescents With Mental Health Problems
Sjöblom, K., Frankenstein, K., Klintwall, L., Nilbrink, J., Zetterqvist, M., Hesser, H., Hedman‐Lagerlöf, E., Gross, J. J., Hellner, C., Bellander, M., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.14871 · Cited by 11 · see citing works
Effect of Parameterization on Statistical Power and Effect Size Estimation in Latent Growth Modeling
Feingold, A. (2021) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2021.1878895 · Cited by 7 · see citing works
Self-citationBoudreaux, B. D., Schenck, J. A., Chu, Z., & Schmidt, M. D. (2022) · Journal for the Measurement of Physical Behaviour · doi:10.1123/jmpb.2021-0060 · Cited by 6 · see citing works
Nath, A., Schimmelpfennig, S., & Konradt, U. (2024) · Occupational Health Science · doi:10.1007/s41542-024-00194-y · Cited by 7 · see citing works
Nijhof, K., Brinke, L. T., Njarðvík, U., & Liber, J. M. (2021) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-020-00761-1 · Cited by 6 · see citing works
Brinke, L. T., Schuiringa, H. D., Menting, A. T. A., Deković, M., Westera, J. J., & de Castro, B. O. (2021) · Cognitive Therapy and Research · doi:10.1007/s10608-021-10261-1 · Cited by 6 · see citing works
Puddephatt, J., Jones, A., Gage, S. H., & Goodwin, L. (2024) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-024-02670-w · Cited by 5 · see citing works
Rosas, C. E., Talavera, G. A., Roesch, S. C., Mendez‐Rodriguez, H., Munoz, F. A., Castañeda, S. F., Mendoza, P., & Gallo, L. C. (2024) · Translational Behavioral Medicine · doi:10.1093/tbm/ibae001 · Cited by 5 · see citing works
Silverman, A. L., Ferguson, I., Bullis, J. R., Bajwa, H., Mei, S., & Beard, C. (2025) · Journal of Mood and Anxiety Disorders · doi:10.1016/j.xjmad.2025.100106 · Cited by 5 · see citing works
Zoellner, L. A., Feeny, N. C., Angula, D. A., Aideed, M. H., Liban, E. N., Egeh, M. H., Awke, A. I., Ismail, A. S., Kunle, M. A., Ali, E., et al. (2023) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2023.101237 · Cited by 8 · see citing works
Effects of a Mind-Body Program for Chronic Pain in Older versus Younger Adults
LaRowe, L. R., Bakhshaie, J., Vranceanu, A., & Greenberg, J. (2023) · Journal of Pain Research · doi:10.2147/jpr.s435639 · Cited by 4 · see citing works
van Gelder, J., Tettero, T., Siezenga, A. M., & Mertens, E. C. A. (2026) · Journal of Medical Internet Research · doi:10.2196/84420 · Cited by 3 · see citing works
Terman, J. M., Rohan, K. J., Stickle, T. R., & Wernhoff, A. (2022) · Behavior Therapy · doi:10.1016/j.beth.2022.10.002 · Cited by 3 · see citing works
Bäckman, L., Weineland, S., Vernmark, K., Radvogin, E., Bjälkebring, P., Enbuske, E., Hermansson, I., Johansson, N., Petersen, N., & Hursti, T. (2026) · Scandinavian Journal of Psychology · doi:10.1111/sjop.70045 · Cited by 2 · see citing works
“The effect size calculated from observed values was Cohen's d = 0.96, while the model-predicted effect size based on the multilevel model was d GMA-raw = 0.86, (Feingold 2019).”
InfluentialBurlew, A. K., Ruglass, L. M., Espinosa, A., McCuistian, C. L., Haeny, A. M., Jordan, A., Roundtree, C., Lopez, J., Vena, A. A., & Morgan‐López, A. A. (2025) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2025.112850 · Cited by 2 · see citing works
Naito, T., Aoyagi, K., Oka, K., & Ishii, K. (2025) · PLoS ONE · doi:10.1371/journal.pone.0337972 · Cited by 2 · see citing works
Janeczko, J. M., Junker, R., Hörter, P., & Holodynski, M. (2024) · Unterrichtswissenschaft · doi:10.1007/s42010-024-00197-2 · Cited by 2 · see citing works
Reinebo, G., Jansson‐Fröjmark, M., Parling, T., Andersson, G., Lindner, P., & Lundgren, T. (2025) · International Journal of Sport and Exercise Psychology · doi:10.1080/1612197x.2025.2531840 · Cited by 3 · see citing works
Glambek, M., Arnestad, M. N., Hoprekstad, Ø. L., Olsen, O. K., & Martinsen, Ø. L. (2025) · Personality and Individual Differences · doi:10.1016/j.paid.2025.113580 · Cited by 1 · see citing works
Wang, Y., Li, M., Xu, H., He, F., Li, M., Wang, L., & Zang, Y. (2026) · BMC Medicine · doi:10.1186/s12916-026-04816-w · Cited by 1 · see citing works
Ruglass, L. M., Espinosa, A., McCuistian, C. L., Haeny, A. M., Lopez, J., Roundtree, C., Vena, A. A., Morgan‐López, A. A., & Burlew, A. K. (2026) · Journal of Substance Use and Addiction Treatment · doi:10.1016/j.josat.2026.210022 · Cited by 1 · see citing works
Feingold, A. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-818697-8.00014-5 · Cited by 1 · see citing works
Self-citationVohl, P., & Loye, N. (2023) · Mesure et évaluation en éducation · doi:10.7202/1113335ar
Vohl, P., & Loye, N. (2023) · Mesure et évaluation en éducation · doi:10.7202/1118377ar
Sjöblom, K., Bellander, M., Nilbrink, J., Zetterqvist, M., Sampaio, F., Hedman‐Lagerlöf, E., Hellner, C., Gross, J. J., Hesser, H., & Bjureberg, J. (2026) · BMJ Open · doi:10.1136/bmjopen-2026-116511
Inkpen, C. S., Tueller, S., & Lattimore, P. K. (2026) · Journal of Experimental Criminology · doi:10.1007/s11292-026-09772-2
- Authors
- Feingold, A.
- Affiliations
- Oregon Social Learning Center (US)
- Corresponding author
- Feingold, Alan
- Venue
- The Quantitative Methods for Psychology
- Published
- 2019-09-01
- Volume
- 15
- Issue
- 2
- Pages
- 96-111
- ISSN
- 2292-1354 (ISSN-L)
- PMC
- PMC7413603
- OpenAlex
- W2972737658
- Cited by
- 34
- References
- 45
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 4 · 2023: 7 · 2022: 3 · 2021: 5 · 2020: 1 · 2019: 1
- FWCI
- 1.67
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- The Quantitative Methods for Psychology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institutes of Health
- National Institute on Alcohol Abuse and Alcoholism (R01AA025069)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- standardized mean difference, Cohen's d, effect size estimation, confidence intervals, standard error bias, delta method, bootstrapping, Mplus, treatment effects, meta-analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
