Using a repeated measures ANOVA to analyze the data from a pretest–posttest design: a potentially confusing task
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
The pretest-posttest control group design (or an extension of it) is a highly prestigious experimental design. A popular analytic strategy involves subjecting the data provided, by this design to a repeated measures analysis of variance (ANOVA). Unfortunately, the statistical results yielded by this type of analysis can easily be misinterpreted, since the score model underlying the analysis is not correct. Examples from recently published articles are used to demonstrate that this statistical procedure has led to (a) incorrect statements regarding treatment effects, (b) completely redundant reanalyses of the same data, and (c) problems with respect to post hoc investigations; Two alternative strategies—gain scores and covariance—are discussed and compared.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Sessions will be videotaped and transcribed, and content analysis will be used to identify and code the major themes in the data [108].”
From LIBER8 design and methods: An integrative intervention for loss of control eating among African American and White adolescent girls (2013) · cites this paper as [108]
Citations
Cites
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 26,755
Experimental Design: Procedures for the Behavioral Sciences
Kirk, R. (2013) · doi:10.4135/9781483384733 · Cited by 8,539
Experimental Design in Psychological Research
(2002) · Foundations of Cognitive Psychology · doi:10.7551/mitpress/3080.003.0011 · Cited by 2,465
Experimental and quasi-experimental designs for research on teaching
Campbell, D. T. (1963) · Medical Entomology and Zoology · Cited by 1,541
Marascuilo, L. A., & Levin, J. R. (1970) · American Educational Research Journal · doi:10.3102/00028312007003397 · Cited by 177
Analysis of Covariance: A Delicate Instrument
ELASHOFF, J. D. (1969) · American Educational Research Journal · doi:10.2307/1161858 · Cited by 176
Leventhal, G. S., Michaels, J. W., & Sanford, C. (1972) · Journal of Personality and Social Psychology · doi:10.1037/h0032877 · Cited by 159
A Comparison of the Precision of Three Experimental Designs Employing a Concomitant Variable
Feldt, L. S. (1958) · Psychometrika · doi:10.1007/bf02289783 · Cited by 111 · cited 1×
“1 For a more detailed comparison of covariance versus gain scores regarding the issue of power, see Feldt (1958) and Gourlay (1953).”
BackgroundType IV errors and interactions.
Levin, J. R., & Marascuilo, L. A. (1972) · Psychological Bulletin · doi:10.1037/h0033455 · Cited by 60
ATIQULLAH, M. (1962) · Biometrika · doi:10.1093/biomet/49.1-2.83 · Cited by 59
Cited by
Items 401–465 of 465
An evaluation of a grade six drug education program
Erickson, D. H. (1984) · Summit (Simon Fraser University)
Bogardus-Groble, M. L. (1989) · University of North Florida Digital Commons (University of North Florida)
Reducing False Hearing In The Elderly: Variability Training In Meta-Audition
Rogers, C. S. (2010) · Open Scholarship Institutional Repository (Washington University in St. Louis) · doi:10.7936/k7rr1wcq
“Repeated-measures ANOVA has been criticized for use in pretestposttest designs (Dimitrov & Rumrill, 2004; Huck & McLean, 1975; Jennings, 1988).”
Readiness to return: psychosocial factors and injury recovery
Mifsud, P. (2005) · Victoria University Research Repository (Victoria University)
Development of a Preferred Bait for Quelea Control
Shumake, S., Gaddis, S. E., & Garrison, M. (1983) · doi:10.1520/stp30175s
List Length and Organization in the Recall of Categorized Lists
Sowder, C. D. (1977) · Psychological Reports · doi:10.2466/pr0.1977.41.3.839
“An additional consideration was that the data subjected to analysis were difference scores or gain scores (Cox, 1957; Huck & McLean, 1975).”
MethodSplit-Plot Designs with Correlated Proportions
Becker, G. (1988) · Psychological Reports · doi:10.2466/pr0.1988.62.3.915
STATISTICAL MODELING OF LONGITUDINAL DATA IN VETERINARY AND MEDICAL SCIENCES
Chaudhary, J. (2015) · International Journal of Livestock Research · doi:10.5455/ijlr.20150125015909
Bookout, J. M. (2010) · ProQuest LLC eBooks · Cited by 2
Toyose, K., Asaba, N., & Nishino, K. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.08.237
An evaluation of a family based drug abuse prevention program
Mullen, R. (1987) · ThinkTech (Texas Tech University)
Covert counterconditioning: A modified desensitization procedure
Simon, G. K. (1985) · ThinkTech (Texas Tech University)
Memory in Parkinson's disease: a feature-processing analysis
Albrecht, J. W. (1987) · ThinkTech (Texas Tech University)
The effect of predictable versus unpredictable shock on rapid eye movement sleep in rats
Clark, R. I. A. (2015) · University of the Witwatersrand, Johannesburg Institutional Repository on DSpace (University of the Witwatersrand, Johannesburg)
Chronic pain and cigarette smoking: The elucidation of a synergistic relationship
Trotter, D. R. M. (2012) · ThinkTech (Texas Tech University)
The Role of Lexical Phonology in Reading
Christowitz, J., Doctor, E. A., & Saling, M. M. (1985) · South African Journal of Psychology · doi:10.1177/008124638501500404
“However, as Huck and McLean (1975) have pointed out, this strategy leads”
BackgroundAmendola, K. L., Weisburd, D. L., Hamilton, E. E., Jones, G., & Slipka, M. G. (2012)
A comparison of brief relaxation therapies: Self- report and physiological indices
Miller, R. K. (1976) · ScholarWorks at University of Montana
Aggression management: A comparison of cognitive restructuring versus social learning techniques
Hafen, G. E. (1981) · uO Research (University of Ottawa) · doi:10.20381/ruor-17516
Nodes of Knowledge: Librarians as Navigators for Geospatial Technology Users
Theobald, R. B., & Vrbancic, E. (2016) · Journal of Map & Geography Libraries · doi:10.1080/15420353.2016.1224794
“The surveys were confidential but not anonymous, to allow comparison of scores before and after the workshop (Huck and McLean 1975).”
BackgroundDiscuss about the Characteristics and Value of Kodaly's Music Teaching
Ji, C. (2016) · DEStech Transactions on Social Science Education and Human Science · doi:10.12783/dtssehs/icaem2016/4257 · Cited by 1
Robinson, E. (2014) · STARS (University of Central Florida)
Brandenburg, I. J. (1977) · OhioLink ETD Center (Ohio Library and Information Network)
Comprehension and Memory of a Sexual Script: A Test of Schema/Scripting Theory.
Kirsch, J. R. (1988) · Louisiana State University · doi:10.31390/gradschool_disstheses.4651
Fear-Induced Self-Control Depletion
Melny, I. (2013) · mediaTUM (Technical University of Munich) · Cited by 1
Utoyo, D. B., Arjadi, R., Ayuningtias, A. U. H., & Jaya, E. S. (2018) · International Conference on Intervention and Applied Psychology (ICIAP) · doi:10.2991/iciap-17.2018.42 · Cited by 1
“This method of analysis is recommended over the normally used repeated measure ANOVA for the pre-test post-test control group experimental design (Dimitrov & Rumrill, 2003; Huck & McLean, 1975).”
MethodA Study on DNA Sequence of Rice Using Scoring Matrix Method and ANOVA Technique
Dutta, A., & Das, K. K. (2018) · Springer proceedings in mathematics & statistics · doi:10.1007/978-981-13-1223-6_2
Integrating Obesity and Eating Disorder Prevention: A Pilot and Feasibility Trial of INSPIRE
Simpson, C. C. (2018) · VCU Scholars Compass (Virginia Commonwealth University) · doi:10.25772/p700-5s34
López, A. T. (2016) · El Repositorio Academico Digital de la UANL (Universidad Autónoma de Nuevo León)
Incorporating the other : investigating body representation and social cognition
Farmer, H. (2014) · Cited by 1
Therapist Mentalization and Patient Outcomes in the First Year of Psychotherapy
Palumbo, M. (2020) · CUNY Academic Works (City University of New York) · Cited by 2
The Impact of Goal Setting and Feedback on the Productivity of Navy Industrial Workers.
Crawford, K. S., White, M. A., & Magnusson, P. (1983) · Defense Technical Information Center (DTIC)
Assessment of the most effective method of teaching cricoid pressure forcePreprint
Whitaker, J., Elsdon, N., Moran, J., Hulme, J., & Peake, S. (2020) · Authorea · doi:10.22541/au.160381881.13145464/v1
The Contribution of Instruction in a Knowledge Mapping Technique to Students' Understanding of Texts
Hussin, M., & Wahab, J. L. A. (2021) · International Journal of Academic Research in Business and Social Sciences · doi:10.6007/ijarbss/v11-i3/8984
“In supporting the above conclusion, Huck and McLean (1975) suggested that the analysis of difference scores was a more conservative procedure than the analysis of covariance.”
ResultHodgson, E. (2020) · Utrecht University Repository (Utrecht University)
Models based on skewed distributions for paired data, with applications to health data
da Cunha, G. Z. (2020) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.17.2020.tde-23082020-143052
Effect of videotape modeling and videotape feedback on job interview behavior
Lovsin, J. V. (1979) · Iowa State University Digital Repository (Iowa State University) · doi:10.31274/rtd-180813-5318
Tu, Y.-K., & Gilthorpe, M. S. (2011) · Chapman & Hall/CRC eBooks · doi:10.1201/b11045-13
Next Week, Next Month, Next Year: How Perceived Temporal Boundaries Affect Initiation ExpectationsPreprint
Hennecke, M., & Converse, B. A. (2016) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/wehbq
Emotional intelligence development via mentoring at a women's college
Jordan, M. K. (2016) · Digital Repository Service (Northeastern University) · doi:10.17760/d20222093
Revisión sistemática sobre el uso de dispositivos ortopédicos en pacientes adultos con pie plano
Olarte, D. M. V., Dulcey, L., León, J. S. T., Sandoval, E. S., Díaz, L. C. S., Zafra, N. M., & Palencia, S. N. H. (2022) · Ciencia Latina Revista Científica Multidisciplinar · doi:10.37811/cl_rcm.v6i3.2357
PALACOL, J. C. C. (2022) · International Journal of Research Publications · doi:10.47119/ijrp1001051720223665
Parrella, J. A., Baker, C. N., Leggette, H. R., & Dunsford, D. W. (2022) · Journal of Applied Communications · doi:10.4148/1051-0834.2444
“Quasi-experimental, pretest-posttest control group designs are advantageous because they are versatile (Joyce, 1975) and designed to fit real-world settings (Koh & Owen, 2000).”
Gracia, F., & Bintari, D. R. (2023) · Journal of Educational Health and Community Psychology · doi:10.12928/jehcp.v1i1.24980 · Cited by 1
Büyüköksüz, E., & Raşit, A. V. C. I. (2023) · Journal of Qualitative Research in Education · doi:10.14689/enad.36.1810
“Pretest, posttest, and design with the control group were used in the study (Huck & McLean 1975; Shadish, Cook, & Campbell, 2002).”
MethodCohen, B. H. (2013) · Wiley eBooks · doi:10.1002/9781394260638.bref
Cohen, B. H. (2013) · Wiley eBooks · doi:10.1002/9781394260638.ch16
Neural Network Impact on Marker Performance in Computer Vision Tasks
Selmanović, E., Mulaimović, E., Delalić, S., Kadrić, Z., & Šabanac, Z. (2024) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro60963.2024.10569478
“Repeated Measures Analysis of Variance (RM ANOVA) [42] was employed to test whether there existed significant differences in the average validation accuracy (average probabilities) among the selected neural networks when classifying sets through of 5 symbols.”
MethodOlder Adults Self-Regulate Learning and Update Knowledge Regarding Psychology-Related Misconceptions
DeCaro, R. E., & Thomas, A. K. (2025) · Experimental Aging Research · doi:10.1080/0361073x.2025.2508100
“If there are no pretest differences, an analysis of gains accounting for pretest scores (e.g., ANCOVA) is the more appropriate analysis (Huck & McLean, 1975; Weinfurt, 2000).”
Arduino Use and its Impact on High School Students’ Programming Success and AttitudesPreprint
ALTAY, G., & Kışla, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5346639
Visual Features Involved in Determining Apparent Elasticity Elicit Touch Desire
Kawabe, T., & Ujitoko, Y. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3590469
“Therefore, we analyzed simple main effects [35] for the significant interaction.”
Büyüköksüz, E., & Erözkan, A. (2025) · International Journal of Cognitive Behavioral Therapy · doi:10.1007/s41811-025-00267-y
“This study used a quasi-experimental design with a pre-test-post-test structure to evaluate the effectiveness of the ACT-BIFA on self-esteem in non-clinical adolescents (Heppner et al., 2008; Huck & McLean, 1975; Valente & MacKinnon, 2017; Wampold, 1986).”
Serdar, K. (2011) · VCU Scholars Compass (Virginia Commonwealth University) · doi:10.25772/7r84-hz95 · Cited by 2
“ANCOVA is a statistical technique that is useful in intervention research because it enables the researcher to control for individual differences in scores at baseline, thus reducing error variance (Huck & McLean, 1975).”
MethodAdditional file 1 of Evidence for foot orthoses for adults with flatfoot: a systematic review
Herchenröder, M., Wilfling, D., & Steinhäuser, J. (2021) · Figshare · doi:10.6084/m9.figshare.17097882
Using Masked Raters to Evaluate Social Validity of a Parent-Implemented Communication Intervention
Akemoğlu, Y., Garcia-Grau, P., & Meadan, H. (2019) · Sage Journals Data · doi:10.25384/sage.c.4610111
Bergen, E. J. (2025) · Digital Commons (Liberty University)
Gudenus, L. M. (2019) · Unipub UB Graz (Universität Graz)
Matematikångest och välmående bland elever i årskurs 7–9
Krokfors, J., & Sandberg, A. (2023) · Doria (University of Helsinki)
Tek Oturum Bilinçli Farkındalık Pratiğinin DRM Paradigmasında Bellek Yanılmaları Üzerindeki Etkisi
KIYAR, E. B. (2023) · Bursa Uludag University - AVESIS
An autoregressive latent change score model for randomized pretest, posttest, follow-up designs
Serang, S., Reese, A. H., & Edmunds, S. R. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-02962-w
EVALUATION OF TELEMENTAL HEALTH CARE WITH VETERANS DURING A PUBLIC HEALTH EMERGENCY
Sang'utei, B. K. (2025) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/aejp-0g79
Seo, K.-J., & Mun, D. (2026) · Computer Applications in Engineering Education · doi:10.1002/cae.70247
Franssen, M., Gao, R., van der Spek, E. D., & Martens, M. H. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2746452
Metadata
- Authors
- Huck, S. W., & McLean, R. A.
- Affiliations
- University of Tennessee at Knoxville (US)
- Venue
- Psychological Bulletin
- Published
- 1975-07
- Volume
- 82
- Issue
- 4
- Pages
- 511-518
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2072435510
- Cited by
- 449
- References
- 19
- Citations by year
- 2026: 4 · 2025: 12 · 2024: 6 · 2023: 8 · 2022: 13 · 2021: 14 · 2020: 18 · 2019: 17 · 2018: 15 · 2017: 11 · 2016: 21 · 2015: 20 · 2014: 15 · 2013: 20 · 2012: 11
- FWCI
- 15.23
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Keywords
- pretest-posttest control group design, repeated measures ANOVA, SCORE risk model, treatment effect estimation, misinterpretation of statistical results, reanalysis of data, post hoc comparisons, gain score, covariance analysis, experimental design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.