The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Single group, pre- and post-test research designs: Some methodological concerns
Abstract
This article provides two illustrations of some of the factors that can influence findings from pre- and post-test research designs in evaluation studies, including regression to the mean (RTM), maturation, history and test effects. The first illustration involves a re-analysis of data from a study by Marsden (), in which pre-test scores are plotted against gain scores to demonstrate RTM effects. The second illustration is a methodological review of single group, pre- and post-test research designs (pre-experiments) that evaluate causal relationships between intervention and outcome. Re-analysis of Marsden’s prior data shows that learners with higher baseline scores consistently made smaller gains than those with lower baseline scores, demonstrating that RTM is clearly observable in single group, pre-post test designs. Our review found that 13% of the sample of 490 articles were evaluation studies. Of these evaluation studies, about half used an experimental design. However, a quarter used a single group, pre-post test design, and researchers using these designs did not mention possible RTM effects in their explanations, although other explanatory factors were mentioned. We conclude by describing how using experimental or quasi-experimental designs would have enabled researchers to explain their findings more accurately, and to draw more useful implications for pedagogy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1963) · Medical Entomology and Zoology · Cited by 15,085
(1979) · Medical Entomology and Zoology · Cited by 11,252
Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis
Norris, J. M., & Ortega, L. (2000) · Language Learning · doi:10.1111/0023-8333.00136 · Cited by 2,569 · cited 1×
“For example, Norris and Ortega’s (2000) meta-analysis of 78 second language education studies found that the average effect sizes of true control and comparison treatment groups was d=0.”
MethodLipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · cited 1×
“A methodological review of studies of psychological, educational and behavioural treatments (Lipsey & Wilson, 1993) showed that the ‘pre- post-test’ design consistently overestimates effectiveness by an average of 61% compared with studies with a control group.”
ResultRegression Towards Mediocrity in Hereditary Stature.
Galton, F. (1886) · The Journal of the Anthropological Institute of Great Britain and Ireland · doi:10.2307/2841583 · Cited by 1,454
Language learning and technology
Healey, D. (2016) · doi:10.4324/9781315657899.ch1 · Cited by 1,419
Effects of Video Club Participation on Teachers' Professional Vision
Gamoran Sherin, M., & van Es, E. A. (2009) · Journal of Teacher Education · doi:10.1177/0022487108328155 · Cited by 1,179
Baker, F., Wigram, T., Stott, D., & McFerran, K. (2008) · Nordic Journal of Music Therapy · doi:10.1080/08098130809478203 · Cited by 1,139
MacCoun, R. (2003) · Journal of Policy Analysis and Management · doi:10.1002/pam.10129 · Cited by 828
Designing Randomised Trials in Health, Education and the Social Sciences
Torgerson, D. J., & Torgerson, C. J. (2008) · doi:10.1057/9780230583993 · Cited by 366
Self-citationEvagorou, M., Korfiatis, K., Nicolaou, C., & Constantinou, C. (2009) · International Journal of Science Education · doi:10.1080/09500690701749313 · Cited by 216 · cited 2×
“Another study (Evagorou et al., 2009) may also illustrate that pre-achievement level can partly explain the gains observed.”
Grace, M. M. (2008) · International Journal of Science Education · doi:10.1080/09500690701744595 · Cited by 204
Podcasting: An Effective Tool for Honing Language Students’ Pronunciation?
Ducate, L., & Lomicka, L. (2009) · Language Learning & Technology · doi:10.64152/10125/44192 · Cited by 179
Brady, S., Gillis, M., Smith, T., Lavalette, M., Liss-Bronstein, L., Lowe, E., North, W., Russo, E., & Wilder, T. D. (2009) · Reading and Writing · doi:10.1007/s11145-009-9166-x · Cited by 169
Annetta, L., Mangrum, J., Holmes, S., Collazo, K., & Cheng, M. (2009) · International Journal of Science Education · doi:10.1080/09500690801968656 · Cited by 166
Further evidence for teacher knowledge: supporting struggling readers in grades three through five
McCutchen, D., Green, L., Abbott, R. D., & Sanders, E. A. (2009) · Reading and Writing · doi:10.1007/s11145-009-9163-0 · Cited by 132
Tsaparlis, G. R., & Papaphotis, G. (2008) · International Journal of Science Education · doi:10.1080/09500690801891908 · Cited by 117
Contribution of Meta‐strategic Knowledge to Scientific Inquiry Learning
Ben‐David, A., & Zohar, A. (2008) · International Journal of Science Education · doi:10.1080/09500690802162762 · Cited by 104
Regression Fallacies in the Matched Groups Experiment
Thorndike, R. L. (1942) · Psychometrika · doi:10.1007/bf02288069 · Cited by 91
Marsden, E. J. (2006) · Language Learning · doi:10.1111/j.1467-9922.2006.00375.x · Cited by 86
Self-citationPark, H., Khan, S. A., & Petrina, S. (2008) · International Journal of Science Education · doi:10.1080/09500690701787891 · Cited by 59
Strategy instruction in writing for adult literacy learners
MacArthur, C. A., & Lembo, L. T. (2008) · Reading and Writing · doi:10.1007/s11145-008-9142-x · Cited by 47
Estimating Linear Size and Scale: Body rulers
Jones, M. G., Taylor, A. R., & Broadwell, B. (2008) · International Journal of Science Education · doi:10.1080/09500690802101976 · Cited by 42
Guisasola, J., Solbes, J., Barragues, J., Pascual, M. M., & Moreno, A. (2008) · International Journal of Science Education · doi:10.1080/09500690802353536 · Cited by 41
Miedijensky, S., & Tal, T. (2008) · International Journal of Science Education · doi:10.1080/09500690802389597 · Cited by 20
O'Byrne, B. (2008) · International Journal of Science Education · doi:10.1080/09500690701730479 · Cited by 18
Graebner, I. T., de Souza, E. M. T., & Saito, C. H. (2008) · International Journal of Science Education · doi:10.1080/09500690701885646 · Cited by 7
Newton, D. P., & Newton, L. D. (2009) · Educational Studies · doi:10.1080/03055690802648119 · Cited by 6
Research works printed but not linked: 8
- 10.1080/01411920802041806 · doi:10.1080/01411920802041806
- Benati A., 2010, Processing instruction and discourse, 97
- 10.1080/08098130809478203 · DOI · Paper page
- 10.1080/0305569980240205 · doi:10.1080/0305569980240205
- 10.1080/09500690801975768 · doi:10.1080/09500690801975768
- Spalding E., 2009, Research in the Teaching of English, 44, 23, 10.58680/rte20097244 · doi:10.58680/rte20097244
- 10.1080/09500690802017545 · doi:10.1080/09500690802017545
- 10.1080/09500690802483093 · doi:10.1080/09500690802483093
Cited by
Interventions in education to prevent STEM pipeline leakage
van den Hurk, A., Meelissen, M., & van Langen, A. (2019) · International Journal of Science Education · doi:10.1080/09500693.2018.1540897 · Cited by 254
A systematic review of AI-based automated written feedback research
Shi, H., & Aryadoust, V. (2024) · ReCALL · doi:10.1017/s0958344023000265 · Cited by 222
Cheang, R., Gillions, A., & Sparkes, E. (2019) · Journal of Child and Family Studies · doi:10.1007/s10826-019-01413-9 · Cited by 95
Taylor, F., & Marsden, E. J. (2014) · The Modern Language Journal · doi:10.1111/modl.12146 · Cited by 70
Self-citationKnight, L., & Samuel, V. (2022) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2022.06.006 · Cited by 37
Metadata
- Authors
- Marsden, E., & Torgerson, C. J.
- Venue
- Oxford Review of Education
- Publisher
- Informa UK Limited
- Published
- 2012-10
- Volume
- 38
- Issue
- 5
- Pages
- 583-616
- ISSN
- 0305-4985 (ISSN-L)
- 1465-3915
- OpenAlex
- W2051211166
- Cited by
- 360
- References
- 28
- FWCI
- 15.44
- Type
- journal article
- Language
- en
- Update notices
- correction (2014-09-16)
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.