The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs.
Abstract
When a meta-analysis on results from experimental studies is conducted, differences in the study design must be taken into consideration. A method for combining results across independent-groups and repeated measures designs is described, and the conditions under which such an analysis is appropriate are discussed. Combining results across designs requires that (a) all effect sizes be transformed into a common metric, (b) effect sizes from each design estimate the same treatment effect, and (c) meta-analysis procedures use design-specific estimates of sampling variance to reflect the precision of the effect size estimates.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Hedges g is viewed as more conservative than ES calculations that have previously been recommended for within-participant data.12,28,39”
In the citing paper
“The advantages of using Hedges g for correlated data (cf.39) include the fact that it controls for small sample size bias9 and does not risk inflation of the overall treatment effect when the data are correlated across time in excess of .50.28”
In the citing paper
“We selected Hedges’ g as our ES based on recent guidelines that indicate it is more conservative than previously recommended ES for use in correlated data, particularly for bias associated with small sample size.12,28,39”
In the citing paper
“Effect sizes from pre-post designs are generally less reliable and less informative about the effects of the intervention because they are likely to be influenced by confounding variables (Morris and DeShon, 2002).”
From The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research (2020) · cites this paper as Morris and DeShon, 2002
In the citing paper
“Generally, and conform to expectations (Light and Pillemer, 1986; Morris and DeShon, 2002), the average effect of feedback as reported in pre-post design studies is higher than in controlled studies and higher reported in published journal articles than in dissertations.”
From The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research (2020) · cites this paper as Morris and DeShon, 2002
In the citing paper
“Effect sizes computed for repeated measures designs and independent groups designs are not directly comparable because different degrees of freedom are used to calculate each [62,63].”
From Menstrual cycle effects on cognitive performance: A meta-analysis (2025) · cites this paper as [63]
In the citing paper
“When computing effect sizes, we did not correct for the correlation between the pre- and post-treatment means despite the within-subjects design (Morris and DeShon 2002); given that we used the LOCF method to deal with missing data, this would have artificially inflated the pre- to post-treatment mean correlations leading to less conservative estimates of effect size.”
From The Effectiveness of Mindfulness-Based Cognitive Therapy (MBCT) in Real-World Healthcare Services (2020) · cites this paper as Morris and DeShon 2002
In the citing paper
“The estimation of treatment effects on the dependent variables, for each study, was based on the calculation of the standardized mean change using raw score standardization (SMCR; see Appendix A), taking the standard deviations of the pretest [33,34].”
From Mindfulness-Based Interventions for People with Schizophrenia: A Systematic Review and Meta-Analysis (2020) · cites this paper as [34]
In the citing paper
“Letting φ() denote the cumulative density function of a standard normal distribution, the interpretation of the standardized mean change can also be facilitated by noting that φ(d) estimates the proportion of individuals for which the difference or change scores reflect a preferable outcome [31].”
From How to boost positive interpretations? A Meta-Analysis of the effectiveness of cognitive bias modification for interpretation (2014) · cites this paper as [31]
In the citing paper
“For an effect size of 0.2 (a “small” effect), the positive interpretation bias is expected to increase (and negative mood is expected to decrease) for φ(0.2) = .58 (i.e., 58%) of individuals.”
From How to boost positive interpretations? A Meta-Analysis of the effectiveness of cognitive bias modification for interpretation (2014) · cites this paper as [31]
In the citing paper
“Effect sizes of 0.5 (a “medium” effect) and 0.8 (a “large” effect) correspond to an increase in positive interpretation bias (and a decrease in negative mood) in 69% and 79% of individuals, respectively [31].”
From How to boost positive interpretations? A Meta-Analysis of the effectiveness of cognitive bias modification for interpretation (2014) · cites this paper as [31]
In the citing paper
“as there was no indication for publication bias [31], we cautiously suggest that even had there been other unpublished data, these would not have changed the results.”
From How to boost positive interpretations? A Meta-Analysis of the effectiveness of cognitive bias modification for interpretation (2014) · cites this paper as [31]
In the citing paper
“Based on the premise for comparing ES previously utilized [58-60], for each of the measures of interest based on the therapeutic intervention (Figure 1).”
Citations
Cites
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
(1979) · Medical Entomology and Zoology · Cited by 11,252
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
Quasi-experimentation: Design and analysis issues for field settings
Shrout, P. E. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90063-4 · Cited by 6,091
Beaton, A. E., & Keppel, G. (1975) · American Educational Research Journal · doi:10.2307/1162588 · Cited by 5,857
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,840
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
Meta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348600 · Cited by 4,249
Research works printed but not linked: 2
- Keppel, 1982, Design and analysis: A researcher's handbook
- 10.1037/0022-006X.63.1.79 · DOI · Paper page
Cited by
Sorted by citations
Items 1,001–1,033 of 1,033
Ying, Y., Ji, Y., Kong, F., Wang, M., Chen, Q., Wang, L., Hou, Y., Yu, L., Zhu, L., Miao, P., et al. (2022) · Psychological Medicine · doi:10.1017/s0033291722000599 · Cited by 31
Boschen, K. A., Robinson, E., Campbell, K. A., Muir, S., Oey, E., Janes, K., Fashler, S. R., & Katz, J. (2016) · Pain Research and Management · doi:10.1155/2016/4678083 · Cited by 29
From the lab to the clinic: effectiveness of cognitive-behavioral treatments for anxiety disorders
van Ingen, D. J., Freiheit, S. R., & Vye, C. S. (2009) · Professional Psychology Research and Practice · doi:10.1037/a0013318 · Cited by 30
Web-based interventions for comorbid depression and chronic illness: a systematic review
Charova, E., Dorstyn, D., Tully, P., & Mittag, O. (2015) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x15571997 · Cited by 30
Lambert, M., Ruppelt, F., Siem, A.-K., Rohenkohl, A. C., Kraft, V., Luedecke, D., Sengutta, M., Schröter, R., Daubmann, A., Correll, C. U., et al. (2018) · Schizophrenia Research · doi:10.1016/j.schres.2017.07.036 · Cited by 29
Young adults with co-occurring disorders: substance use disorder treatment response and outcomes
Bergman, B. G., Greene, M. C., Slaymaker, V., Hoeppner, B. B., & Kelly, J. F. (2014) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2013.11.005 · Cited by 28
Kinnunen, S. M., Puolakanaho, A., Tolvanen, A., Mäkikangas, A., & Lappalainen, R. (2019) · International Journal of Stress Management · doi:10.1037/str0000095 · Cited by 28
Blatz, C. W., & Mercier, B. (2018) · Social Psychological and Personality Science · doi:10.1177/1948550617712034 · Cited by 33
Scharff, A., Ortiz, S. N., Forrest, L. N., Smith, A. R., & Boswell, J. F. (2021) · Journal of Clinical Psychology · doi:10.1002/jclp.23106 · Cited by 27
Du Puy, R. S., Poortvliet, R. K. E., Mooijaart, S. P., den Elzen, W. P. J., Jagger, C., Pearce, S. H. S., Arai, Y., Hirose, N., Teh, R., Menzies, O., et al. (2021) · Thyroid® · doi:10.1089/thy.2020.0567 · Cited by 27
Treatments for Co-Occurring Depression and Substance Use in Young People: A Systematic Review
Deady, M., Teesson, M., & Kay-Lambkin, F. (2014) · Current Drug Abuse Reviews · doi:10.2174/1874473707666141015220608 · Cited by 25
Vowles, K. E., Sowden, G., Hickman, J., & Ashworth, J. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2018.10.012 · Cited by 24
Bentley, K. H., Sauer-Zavala, S., Stevens, K. T., & Washburn, J. J. (2020) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2018.09.012 · Cited by 28
Intensive Multidisciplinary Treatment of Severe Somatoform Disorder: A Prospective Evaluation
Houtveen, J. H., van Broeckhuysen-Kloth, S., Lintmeijer, L. L., Bühring, M. E. F., & Geenen, R. (2015) · Journal of Nervous & Mental Disease · doi:10.1097/nmd.0000000000000250 · Cited by 24
Graham, C. D., O'Hara, D. J., & Kemp, S. (2018) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2174 · Cited by 24
Kinnunen, S. M., Puolakanaho, A., Mäkikangas, A., Tolvanen, A., & Lappalainen, R. (2020) · International Journal of Stress Management · doi:10.1037/str0000132 · Cited by 23
Kremer, F., Ditzen, B., & Wischmann, T. (2023) · PLoS ONE · doi:10.1371/journal.pone.0282065 · Cited by 22
“Given the available data, an effect size calculation was performed for mean differences of groups with unequal sample sizes within a pre-post control design as described by Morris and DeShon [37,38].”
MethodRetractedRalston, G. W., Kilgore, L., Wyatt, F. B., Dutheil, F., Jaekel, P., Buchan, D. S., & Baker, J. S. (2019) · Frontiers in Physiology · doi:10.3389/fphys.2019.00864 · Cited by 15
Aazh, H., Bryant, C., & Moore, B. C. J. (2019) · American Journal of Audiology · doi:10.1044/2019_aja-19-0045 · Cited by 22
Menstrual cycle effects on cognitive performance: A meta-analysis
Jang, D., Zhang, J., & Elfenbein, H. A. (2025) · PLOS ONE · doi:10.1371/journal.pone.0318576 · Cited by 21
Impact of a blended web-based mindfulness programme for general practitioners: a pilot study
Montero-Marin, J., Gaete, J., Araya, R., Demarzo, M., Manzanera, R., Álvarez de Mon, M., & García-Campayo, J. (2018) · Mindfulness · doi:10.1007/s12671-017-0752-8 · Cited by 22
Efficacy and safety of varenicline for smoking cessation in schizophrenia: a meta-analysis.
Ahmed, S., Virani, S., Kotapati, V. P., Bachu, R., Adnan, M., Khan, A. M., Zubair, A., Begum, G., Kumar, J., Qureshi, M., et al. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00428 · Cited by 20
Kohtala, A., Muotka, J., & Lappalainen, R. (2017) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2017.03.003 · Cited by 16
Rebello, C. J., Zhang, D., Kirwan, J. P., Lowe, A. C., Emerson, C. J., Kracht, C. L., Steib, L. C., Greenway, F. L., Johnson, W. D., & Brown, J. C. (2023) · International Journal of Obesity · doi:10.1038/s41366-023-01283-8 · Cited by 16
“The SMD denotes the difference between the mean glucose values of the control and exercise groups, divided by the pooled standard deviation [23].”
Singer, A., Wolf, M., Generoso, L., Arias, E., Delcastillo, K., Echevarria, E., Martinez, A., Androulakis Korakakis, P., Refalo, M. C., Swinton, P. A., et al. (2024) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2024.1429789 · Cited by 14
Fristad, M. A., Ackerman, J. P., & Nick, E. A. (2018) · Evidence-Based Practice in Child and Adolescent Mental Health · doi:10.1080/23794925.2018.1509031 · Cited by 14
Forst, D. A., Podgurski, A. F., Strander, S. M., Whitman, J. D., Datta, S., Pintro, K., Horick, N. K., Willis, K. D., Sannes, T. S., Greer, J. A., et al. (2024) · Journal of Clinical Oncology · doi:10.1200/jco.24.00065 · Cited by 13
Melendez-Torres, G. J., Thomas, J., Lorenc, T., O’Mara-Eves, A., & Petticrew, M. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0821-7 · Cited by 12
Feliu‐Soler, A., Pérez‐Aranda, A., Andrés‐Rodríguez, L., Butjosa, A., Díaz, N. S., Trujols, J., Núñez, C., Stephan‐Otto, C., Rozadilla‐Sacanell, A., Serrano‐Blanco, A., et al. (2018) · Journal of Applied Biobehavioral Research · doi:10.1111/jabr.12134 · Cited by 12
Burgos‐Julián, F. A., Ruiz‐Íñiguez, R., Peña‐Ibáñez, F., Montero, A. C., & Santed‐Germán, M. A. (2022) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2717 · Cited by 10
Cognitive behavioral therapy for frequent attenders in primary care
Strömbom, Y., Karlsson, J., Fredrikson, M., Melin, L., & Magnusson, P. (2018) · Health Science Reports · doi:10.1002/hsr2.80 · Cited by 4
“Model‐based standarized effect sizes for within‐subject effects were calculated by dividing the absolute treatment effect by the within‐subject (residual) standard deviation.(32) We chose the random intercept mixed model because it allows testing for both time and treatment effects, in situations with repeated measure…”
MethodYu, Y., Zhao, W., Li, A., Shanks, D. R., Hu, X., Luo, L., & Yang, C. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10024-4 · Cited by 2
Adolescents’ dual affective response to dance in training, grading, and competition
Szabo, A., Takács, D., Magyar, M., Soós, I., Boros, S., & Bősze, J. (2020) · Current Psychology · doi:10.1007/s12144-018-9800-3 · Cited by 2
Metadata
- Authors
- Morris, S. B., & DeShon, R. P.
- Affiliations
- Illinois Institute of Technology (US)
- Michigan State University (US)
- Venue
- Psychological Methods
- Published
- 2002
- Volume
- 7
- Issue
- 1
- Pages
- 105-125
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2076029586
- Semantic Scholar
- d88e7804b7df89d229fd5db7a1c567b94ff18f33
- Cited by
- 2,502
- References
- 54
- Citations by year
- 2026: 182 · 2025: 116 · 2024: 90 · 2023: 138 · 2022: 137 · 2021: 148 · 2020: 180 · 2019: 211 · 2018: 219 · 2017: 190 · 2016: 135 · 2015: 147 · 2014: 148 · 2013: 124 · 2012: 81
- FWCI
- 13.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- meta-analysis, repeated measures designs, treatment effect estimation, experimental design
- MeSH terms
- Humans, Models, Psychological, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
