Randomised Controlled Trials for Evidence-based Education: Some Problems in Judging 'What Works'
The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
The rise of evidence-based practices in education seeks to base policy making on randomised controlled trials (RCTs) and meta-analysis, rather than simply political will, in determining 'what works' in education. This paper argues that the faith that is placed in RCTs to determine 'what works' in education may be misplaced. The rigour required for RCTs may not be practicable, and evidence-based practices in education that are based largely on RCTs face several problems as they: (a) operate from a restricted view of causality and predictability; (b) understate the value of other data sources; (c) neglect the significance of theories of chaos and complexity; (d) display unrealistic reductionism, simplification and atomisation of a complex whole; (e) underestimate the importance of multiple perspectives on 'what works'; (f) are unable to catch the dynamics of non-linear phenomena; (g) are unable to comment on the processes taking place in experiments; (h) neglect the significance of context. Judging 'what works' is a deliberative and evaluative as well as an empirical matter.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Cohen, L., Manion, L., & Morrison, K. (2007) · Routledge eBooks · doi:10.4324/9780203029053 · Cited by 20,105
Self-citationThe fifth discipline, the art and practice of the learning organization
Senge, P. M. (1991) · Performance + Instruction · doi:10.1002/pfi.4170300510 · Cited by 20,060
(1963) · Medical Entomology and Zoology · Cited by 15,085
Meta-analysis in social research
(1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90011-8 · Cited by 3,853 · cited 4×
“For a discussion of effect size and statistical signifi- cance in meta-analysis see Carver (1978), Coe (2000), Fitz-Gibbon (1997), Glass et al. (1981), Lipsey (1992), Thompson (1994, 1996, 1998), Thompson & Snyder (1997) and Tymms (1999).”
BackgroundMethodInfluentialEvaluation: A Systematic Approach
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (1979) · Cited by 2,943
Meta-analysis of psychotherapy outcome studies.
Smith, M. L., & Glass, G. V. (1977) · American Psychologist · doi:10.1037/0003-066x.32.9.752 · Cited by 2,154 · cited 3×
“The latter argue that, nonetheless, weak studies can add up to a strongconclusion (Smith & Glass, 1977:221) and that the differences in the size of experimental effects between high-validity and low-validity studies are surprisingly small (Smith & Glass, 1977: 226).4 If meta-analysis is to be fair,…”
BackgroundBest evidence synthesis: An intelligent alternative to meta-analysis
Slavin, R. E. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00097-a · Cited by 949 · cited 1×
“…there needs to be differentiation between ‘good’ and ‘bad’ RCTs – between methodologicallyrigorous and poorly constructed RCTs (Cook et al., 1992: 297), a point which has been addressed by Evans et al. (2000), Glass et al. (1981), Leva i and Glatter (2000), Slavin (1995) and Smith and Glass (1977).”
BackgroundGeach, P. T. (1956) · Analysis · doi:10.1093/analys/17.2.33 · Cited by 699
Best-Evidence Synthesis: An Alternative to Meta-Analytic and Traditional Reviews
Slavin, R. E. (1986) · Educational Researcher · doi:10.2307/1174711 · Cited by 698 · cited 2×
“…naturalistic settings such as schools and classrooms are not the antiseptic, reductionist, analysed-out or analysable-out world of the laboratory (Slavin, 1986), and that the degree of control required for experimental conditions to be met renders classrooms unnatural settings, yet the…”
BackgroundWhat is Evidence-based Education?
Davies, P. (1999) · British Journal of Educational Studies · doi:10.1111/1467-8527.00106 · Cited by 650 · cited 12×
“For many advocates of EBE, the nature of evidence is of a particular kind only, and ‘trying it out’ is also only undertaken in particular ways: data are to be derived from randomised controlled trials (RCTs) – the controlled experiment with randomisation to ensure generalisability (e.g. Boruch, 1997; Oakley, 1998,2000…”
BackgroundInfluential
Cited by
Combining Methods in Educational and Social Research
Gorard, S. A. C., & Taylor, C. (2004) · Cited by 458
A systematic review of social marketing effectiveness
Stead, M., Gordon, R., Angus, K., & McDermott, L. (2007) · Health Education · doi:10.1108/09654280710731548 · Cited by 429
“…cities or communities are used; it is also difficult, where effects are found, to identify whether these are attributable to particular intervention components, or to the combination of activities, or to other factors such as secular trends (Wolff, 2000; Fawcett et al, 1997; Morrison, 2001).”
BackgroundLanguage teacher research engagement
Borg, S. (2010) · Language Teaching · doi:10.1017/s0261444810000170 · Cited by 418
“For further perspectives on this debate, see Hammersley (2001, 2004b), Davies (2000), Maxwell (2004), Morrison (2001) and Feuer, Towne & Shavelson (2002).”
BackgroundEducational Philosophy and the Challenge of Complexity Theory
Morrison, K. (2008) · Educational Philosophy and Theory · doi:10.1111/j.1469-5812.2007.00394.x · Cited by 287
“Even if we could conduct an experiment, its applicability to ongoing, emerging, interactive, relational, open situations, in practice, is limited (Morrison, 2001).”
BackgroundSelf-citationImplementation Research: Finding Common Ground on What, How, Why, Where, and Who
Century, J., & Cassata, A. (2016) · Review of Research in Education · doi:10.3102/0091732x16665332 · Cited by 220
“In medical clinical settings, “what works” is a matter of judgment that weighs, judges, and considers RCT findings in context (Morrison, 2001).”
Background
Metadata
- Authors
- Morrison, K.
- Corresponding author
- Morrison, Keith
- ORCID
- Morrison, Keith
- Venue
- Evaluation & Research in Education
- Publisher
- Informa UK Limited
- Published
- 2001-10
- Volume
- 15
- Issue
- 2
- Pages
- 69-83
- ISSN
- 0950-0790 (ISSN-L)
- 1747-7514
- OpenAlex
- W1995225970
- Cited by
- 90
- References
- 23
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 1 · 2023: 4 · 2022: 2 · 2021: 4 · 2020: 6 · 2019: 5 · 2018: 3 · 2017: 9 · 2016: 9 · 2015: 4 · 2014: 7 · 2013: 3 · 2012: 1
- FWCI
- 3.98
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Evaluation & Research in Education (publishedVersion)
- Topics
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Social Sciences › Education › Educational Research and Methods
- Social Sciences › Education › Educational Assessment and Improvement
- Keywords
- evidence-based education, randomized controlled trials, education policy, causality, complexity theory, chaos theory, reductionism, multiple perspectives, non-linear phenomena, meta-analysis, scientific rigor
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Randomised Controlled Trials for Evidence-based Education: Some Problems in Judging 'What Works'. https://app.cochat.ai/papers/10.1080/09500790108666984
- The paper
- Morrison, K. (2001). Randomised Controlled Trials for Evidence-based Education: Some Problems in Judging 'What Works'. Evaluation & Research in Education. https://doi.org/10.1080/09500790108666984
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.