The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs.
Abstract
When a meta-analysis on results from experimental studies is conducted, differences in the study design must be taken into consideration. A method for combining results across independent-groups and repeated measures designs is described, and the conditions under which such an analysis is appropriate are discussed. Combining results across designs requires that (a) all effect sizes be transformed into a common metric, (b) effect sizes from each design estimate the same treatment effect, and (c) meta-analysis procedures use design-specific estimates of sampling variance to reflect the precision of the effect size estimates.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Hedges g is viewed as more conservative than ES calculations that have previously been recommended for within-participant data.12,28,39”
In the citing paper
“The advantages of using Hedges g for correlated data (cf.39) include the fact that it controls for small sample size bias9 and does not risk inflation of the overall treatment effect when the data are correlated across time in excess of .50.28”
In the citing paper
“We selected Hedges’ g as our ES based on recent guidelines that indicate it is more conservative than previously recommended ES for use in correlated data, particularly for bias associated with small sample size.12,28,39”
In the citing paper
“Effect sizes from pre-post designs are generally less reliable and less informative about the effects of the intervention because they are likely to be influenced by confounding variables (Morris and DeShon, 2002).”
From The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research (2020) · cites this paper as Morris and DeShon, 2002
In the citing paper
“Generally, and conform to expectations (Light and Pillemer, 1986; Morris and DeShon, 2002), the average effect of feedback as reported in pre-post design studies is higher than in controlled studies and higher reported in published journal articles than in dissertations.”
From The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research (2020) · cites this paper as Morris and DeShon, 2002
In the citing paper
“Effect sizes computed for repeated measures designs and independent groups designs are not directly comparable because different degrees of freedom are used to calculate each [62,63].”
From Menstrual cycle effects on cognitive performance: A meta-analysis (2025) · cites this paper as [63]
In the citing paper
“When computing effect sizes, we did not correct for the correlation between the pre- and post-treatment means despite the within-subjects design (Morris and DeShon 2002); given that we used the LOCF method to deal with missing data, this would have artificially inflated the pre- to post-treatment mean correlations leading to less conservative estimates of effect size.”
From The Effectiveness of Mindfulness-Based Cognitive Therapy (MBCT) in Real-World Healthcare Services (2020) · cites this paper as Morris and DeShon 2002
In the citing paper
“The estimation of treatment effects on the dependent variables, for each study, was based on the calculation of the standardized mean change using raw score standardization (SMCR; see Appendix A), taking the standard deviations of the pretest [33,34].”
From Mindfulness-Based Interventions for People with Schizophrenia: A Systematic Review and Meta-Analysis (2020) · cites this paper as [34]
In the citing paper
“Letting φ() denote the cumulative density function of a standard normal distribution, the interpretation of the standardized mean change can also be facilitated by noting that φ(d) estimates the proportion of individuals for which the difference or change scores reflect a preferable outcome [31].”
From How to boost positive interpretations? A Meta-Analysis of the effectiveness of cognitive bias modification for interpretation (2014) · cites this paper as [31]
In the citing paper
“For an effect size of 0.2 (a “small” effect), the positive interpretation bias is expected to increase (and negative mood is expected to decrease) for φ(0.2) = .58 (i.e., 58%) of individuals.”
From How to boost positive interpretations? A Meta-Analysis of the effectiveness of cognitive bias modification for interpretation (2014) · cites this paper as [31]
In the citing paper
“Effect sizes of 0.5 (a “medium” effect) and 0.8 (a “large” effect) correspond to an increase in positive interpretation bias (and a decrease in negative mood) in 69% and 79% of individuals, respectively [31].”
From How to boost positive interpretations? A Meta-Analysis of the effectiveness of cognitive bias modification for interpretation (2014) · cites this paper as [31]
In the citing paper
“as there was no indication for publication bias [31], we cautiously suggest that even had there been other unpublished data, these would not have changed the results.”
From How to boost positive interpretations? A Meta-Analysis of the effectiveness of cognitive bias modification for interpretation (2014) · cites this paper as [31]
In the citing paper
“Based on the premise for comparing ES previously utilized [58-60], for each of the measures of interest based on the therapeutic intervention (Figure 1).”
Citations
Cites
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
(1979) · Medical Entomology and Zoology · Cited by 11,252
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
Quasi-experimentation: Design and analysis issues for field settings
Shrout, P. E. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90063-4 · Cited by 6,091
Beaton, A. E., & Keppel, G. (1975) · American Educational Research Journal · doi:10.2307/1162588 · Cited by 5,857
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,840
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
Meta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348600 · Cited by 4,249
Meta-analysis in social research
(1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90011-8 · Cited by 3,896
Fixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings.
Raudenbush, S. W., Hunter, J. E., & Schmidt, F. L. (1991) · Journal of the American Statistical Association · doi:10.2307/2289738 · Cited by 2,368
Book Review: Design and Analysis: A Researcher's Handbook
Aaker, D. A., & Keppel, G. (1976) · Journal of Marketing Research · doi:10.2307/3150757 · Cited by 2,033
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033
Statistical Principles in Experimental Design.
Geisser, S., & Winer, B. J. (1963) · Journal of the American Statistical Association · doi:10.2307/2283359 · Cited by 1,934
Meta-analysis of experiments with matched groups or repeated measures designs
Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.170 · Cited by 1,693 · cited 7×
“Others have suggested pooling pre- and posttest standard deviations rather than using the pretest standard deviation in Equation 13 (Dunlap et al., 1996; Taylor & White, 1992).”
BackgroundMethodInfluentialGender and the evaluation of leaders: A meta-analysis.
Eagly, A. H., Makhijani, M. G., & Klonsky, B. G. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.1.3 · Cited by 1,657
The self-reference effect in memory: A meta-analysis
Symons, C. S., & Johnson, B. T. (1997) · Psychological Bulletin · doi:10.1037/0033-2909.121.3.371 · Cited by 1,361 · cited 1×
“For example, in a meta-analysis on the effects of self-reference on memory (Symons & Johnson, 1997), studies using a single-group pretest–posttest design produced a mean effect size that was very close to the estimate from studies that included a control group.”
MethodHandbook of Research Methods in Social and Personality Psychology: Author Index
Reis, H. T., & Judd, C. M. (2013) · Cambridge University Press eBooks · doi:10.1017/cbo9780511996481 · Cited by 1,080
Estimation of effect size from a series of independent experiments.
Hedges, L. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.92.2.490 · Cited by 1,030
Synthesizing standardized mean‐change measures
Becker, B. J. (1988) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1988.tb00901.x · Cited by 889
Identifying effective psychological treatments for insomnia: A meta-analysis.
Murtagh, D. R. R., & Greenwood, K. M. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.63.1.79 · Cited by 664
Handbook of Research Methods. in Social and Personality Psychology
Lai, H. L. T. (2000) · Cited by 629
THE EFFECTS OF PSYCHOLOGICALLY BASED INTERVENTION PROGRAMS ON WORKER PRODUCTIVITY: A META‐ANALYSIS
Guzzo, R. A., JETTE, R. D., & Katzell, R. A. (1985) · Personnel Psychology · doi:10.1111/j.1744-6570.1985.tb00547.x · Cited by 520
Gist, M. E., Stevens, C. K., & Bavetta, A. G. (1991) · Personnel Psychology · doi:10.1111/j.1744-6570.1991.tb00701.x · Cited by 502
A cumulative study of the effectiveness of managerial training.
Burke, M. J., & Day, R. R. (1986) · Journal of Applied Psychology · doi:10.1037/0021-9010.71.2.232 · Cited by 500
The effects of psychological therapies under clinically representative conditions: a meta-analysis
Shadish, W. R., Navarro, A. M., Matt, G. E., & Phillips, G. (2000) · Psychological Bulletin · doi:10.1037/0033-2909.126.4.512 · Cited by 381
Cortina, J. M., & Nouri, H. (2000) · Sage eBooks · doi:10.4135/9781412984010 · Cited by 238
Estimation of Effect Size From a Series of Experiments Involving Paired Comparisons
Gibbons, R. D., Hedeker, D., & Davis, J. M. (1993) · Journal of Educational Statistics · doi:10.3102/10769986018003271 · Cited by 230
Distribution of the standardized mean change effect size for meta‐analysis on repeated measures
Morris, S. B. (2000) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711000159150 · Cited by 206
Self-citationImpact of experimental design on effect size: Findings from the research literature on training.
Carlson, K. D., & Schmidt, F. L. (1999) · Journal of Applied Psychology · doi:10.1037/0021-9010.84.6.851 · Cited by 203
Neuman, G. A., EDWARDS, J. E., & Raju, N. S. (1989) · Personnel Psychology · doi:10.1111/j.1744-6570.1989.tb00665.x · Cited by 187
Fleishman, E. A., & Hempel, W. E. (1955) · Journal of Experimental Psychology · doi:10.1037/h0044697 · Cited by 170
Quantitative synthesis of social psychological research.
Johnson, B. T., & Eagly, A. H. (2000) · OpenCommons - UConn (University of Connecticut) · Cited by 168
Meta-analytic evaluation of skills training research for individuals with severe mental illness.
Dilk, M. N., & Bond, G. R. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.64.6.1337 · Cited by 157
TRAINING TEAM PERFORMANCE‐RELATED ASSERTIVENESS
Smith‐Jentsch, K. A., Salas, E., & BAKER, D. P. (1996) · Personnel Psychology · doi:10.1111/j.1744-6570.1996.tb02454.x · Cited by 157
How interchangeable are different estimators of effect size?
Ray, J. W., & Shadish, W. R. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.6.1316 · Cited by 137
Covariance adjustment versus gain scores--revisited.
Maris, E. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.3.309 · Cited by 107
Meta-analytic evaluation of skills training research for individuals with severe mental illness.
Dilk, M. N., & Bond, G. R. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.6.1337 · Cited by 107
SUPERVISORY RELATIONSHIPS TRAINING: A BEHAVIORAL EVALUATION OF A BEHAVIOR MODELING PROGRAM
Moses, J. L., & Ritchie, R. J. (1976) · Personnel Psychology · doi:10.1111/j.1744-6570.1976.tb00417.x · Cited by 69
Li, Z., & Begg, C. (1994) · Journal of the American Statistical Association · doi:10.1080/01621459.1994.10476892 · Cited by 65
Estimation of Effect Size from a Series of Experiments Involving Paired Comparisons
Gibbons, R. D., Hedeker, D., & Davis, J. M. (1993) · Journal of Educational Statistics · doi:10.2307/1165136 · Cited by 64
The Managerial Performance Implications of a Developmental Assessment Center Process
Engelbrecht, A. S., & Fischer, A. H. (1995) · Human Relations · doi:10.1177/001872679504800405 · Cited by 58
MANAGEMENT MODELING TRAINING TO IMPROVE MORALE AND CUSTOMER SATISFACTION1
Smith, P. E. (1976) · Personnel Psychology · doi:10.1111/j.1744-6570.1976.tb00419.x · Cited by 56
EVALUATION OF AN INTERVIEWEE SKILLS TRAINING PROGRAM IN A NATURAL FIELD EXPERIMENT
Campion, M. A., & Campion, J. E. (1987) · Personnel Psychology · doi:10.1111/j.1744-6570.1987.tb00619.x · Cited by 48
Consistency of performance under stable conditions of work.
Rambo, W. W., Chomiak, A. M., & Price, J. M. (1983) · Journal of Applied Psychology · doi:10.1037/0021-9010.68.1.78 · Cited by 45
A field experiment validating contingency model leadership training.
Fiedler, F. E., & Mahar, L. (1979) · Journal of Applied Psychology · doi:10.1037/0021-9010.64.3.247 · Cited by 40
Human relations training and the stability of leadership patterns.
Harris, E. F., & Fleishman, E. A. (1955) · Journal of Applied Psychology · doi:10.1037/h0046585 · Cited by 39
Correcting effect sizes computed from factor analysis of variance for use in meta-analysis.
Morris, S. B., & DeShon, R. P. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.2.192 · Cited by 37
Self-citationVALIDATION OF CONTINGENCY MODEL LEADERSHIP TRAINING: LEADER MATCH.
Leister, A. F., Borden, D., & Fiedler, F. E. (1977) · Academy of Management Journal · doi:10.2307/255420 · Cited by 36
Determining relative importance of predictors with the observational design.
Cortina, J. M., & DeShon, R. P. (1998) · Journal of Applied Psychology · doi:10.1037/0021-9010.83.5.798 · Cited by 34
Self-citationEffects of Mental Practice and Physical Practice upon Muscular Endurance
Kelsey, I. B. (1961) · Research Quarterly American Association for Health Physical Education and Recreation · doi:10.1080/10671188.1961.10762070 · Cited by 29
Investigation of Interpersonal Management Training for Educational Administrators
Smith, R. M., Montello, P. A., & White, P. (1992) · The Journal of Educational Research · doi:10.1080/00220671.1992.9941122 · Cited by 17
An Evaluation of Alternative Methods for Computing Standardized Mean Difference Effect Size
Taylor, M. J., & White, K. R. (1992) · The Journal of Experimental Education · doi:10.1080/00220973.1992.9943850 · Cited by 14
Li, Z., & Begg, C. B. (1994) · Journal of the American Statistical Association · doi:10.2307/2291015 · Cited by 10
Research works printed but not linked: 2
- Keppel, 1982, Design and analysis: A researcher's handbook
- 10.1037/0022-006X.63.1.79 · DOI · Paper page
Cited by
Sorted by citations
Lakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303
Roberts, B. W., Walton, K. E., & Viechtbauer, W. (2006) · Psychological Bulletin · doi:10.1037/0033-2909.132.1.1 · Cited by 3,318
Unemployment impairs mental health: Meta-analyses
Paul, K. I., & Moser, K. (2009) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2009.01.001 · Cited by 2,706
Cook, D. A., Hatala, R., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., Erwin, P. J., & Hamstra, S. J. (2011) · JAMA · doi:10.1001/jama.2011.1234 · Cited by 2,170
Estimating effect sizes from pretest-posttest-control group designs
Morris, S. B. (2008) · Organizational Research Methods · doi:10.1177/1094428106291059 · Cited by 2,133
Self-citation
Metadata
- Authors
- Morris, S. B., & DeShon, R. P.
- Affiliations
- Illinois Institute of Technology (US)
- Michigan State University (US)
- Venue
- Psychological Methods
- Published
- 2002
- Volume
- 7
- Issue
- 1
- Pages
- 105-125
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2076029586
- Semantic Scholar
- d88e7804b7df89d229fd5db7a1c567b94ff18f33
- Cited by
- 2,502
- References
- 54
- Citations by year
- 2026: 182 · 2025: 116 · 2024: 90 · 2023: 138 · 2022: 137 · 2021: 148 · 2020: 180 · 2019: 211 · 2018: 219 · 2017: 190 · 2016: 135 · 2015: 147 · 2014: 148 · 2013: 124 · 2012: 81
- FWCI
- 13.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- meta-analysis, repeated measures designs, treatment effect estimation, experimental design
- MeSH terms
- Humans, Models, Psychological, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
