Cited by 234Meta-analysis and systematic reviewsSchool Choice and PerformanceEducational Research and Methods

The DOI matches its Crossref record; 43 of the 44 references it cites were found in a registry or an index.

Rigorous Large-Scale Educational RCTs Are Often Uninformative: Should We Be Concerned?

Open access

Lortie-Forgues, Hugues, Inglis, Matthew·2019·Educational Researcher

DOI 10.3102/0013189x19832850

Open this paper in CoChat — free account Open PDF

Abstract

There are a growing number of large-scale educational randomized controlled trials (RCTs). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale RCTs commissioned by the UK-based Education Endowment Foundation and the U.S.-based National Center for Educational Evaluation and Regional Assistance, which evaluated interventions aimed at improving academic achievement in K–12 (141 RCTs; 1,222,024 students). The mean effect size was 0.06 standard deviations. These sat within relatively large confidence intervals (mean width = 0.30 SDs), which meant that the results were often uninformative (the median Bayes factor was 0.56). We argue that our field needs, as a priority, to understand why educational RCTs often find small and uninformative effects.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.3102/0013189x19832850

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Introduction to Meta‐Analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Introduction to Meta‐Analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Untitled work
(2000) · Cited by 9,198
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 4×
“Unfavorable findings from traditional research are less likely to be published (Rosenthal, 1979), and many researchers selectively report analyses and conduct unplanned analyses (John, Lowenstein, & Prelec, 2012).”
BackgroundResultInfluential
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · cited 3×
“Recent developments, collectively referred to as the ‘replication crisis’, suggest that the psychological literature is not as robust as previously imagined (e.g., Open Science Collaboration, 2015).”
BackgroundResult
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567

Show all 44 references

Research works printed but not linked: 5

Education Endowment Foundation, 2015, Annual report 2014/15
Education Endowment Foundation, 2015, EEF guidance on cost evaluation
Education Endowment Foundation, 2016, Classification of the security of findings from EEF evaluations
Education Endowment Foundation, 2017, EEF standards for independent evaluation panel members
Jeffreys H., 1961, Theory of probability

Sorted by citations

Interpreting effect sizes of education interventions
Kraft, M. A. (2020) · Educational Researcher · doi:10.3102/0013189x20912798 · Cited by 1,281
Post Qualitative Inquiry, the Refusal of Method, and the Risk of the New
Pierre, E. A. S. (2019) · Qualitative Inquiry · doi:10.1177/1077800419863005 · Cited by 313
Beyond small, medium, or large: points of consideration when interpreting effect sizes
Bakker, A., Cai, J., English, L., Kaiser, G., Mesa, V., & Van Dooren, W. (2019) · Educational Studies in Mathematics · doi:10.1007/s10649-019-09908-4 · Cited by 302
Scaling up behavioral science interventions in online education
Kizilcec, R. F., Reich, J., Yeomans, M., Dann, C., Brunskill, E., Lopez, G., Turkay, S., Williams, J. J., & Tingley, D. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1921417117 · Cited by 208
Achievement Gaps in the Wake of COVID-19
Bailey, D. H., Duncan, G. J., Murnane, R. J., & Au Yeung, N. (2021) · Educational Researcher · doi:10.3102/0013189x211011237 · Cited by 134
Tackling Educational Inequalities with Social Psychology: Identities, Contexts, and Interventions
Easterbrook, M. J., & Hadden, I. R. (2021) · Social Issues and Policy Review · doi:10.1111/sipr.12070 · Cited by 110
Building Content Knowledge to Boost Comprehension in the Primary Grades
Cabell, S. Q., & Hwang, H. (2020) · Reading Research Quarterly · doi:10.1002/rrq.338 · Cited by 104
Both Questionable and Open Research Practices Are Prevalent in Education Research
Makel, M. C., Hodges, J., Cook, B. G., & Plucker, J. A. (2021) · Educational Researcher · doi:10.3102/0013189x211001356 · Cited by 97
Improving student achievement through professional development: Results from a randomised controlled trial of Quality Teaching Rounds
Gore, J. M., Miller, A., Fray, L., Harris, J., & Prieto, E. (2021) · Teaching and Teacher Education · doi:10.1016/j.tate.2021.103297 · Cited by 90
Educational research: an unorthodox introduction
Thomas, G. (2020) · Educational Review · doi:10.1080/00131911.2020.1813944 · Cited by 84
Training spatial cognition enhances mathematical learning in a randomized study of 17,000 children
Judd, N., & Klingberg, T. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01118-4 · Cited by 92
Relevance of Educational Research: An Ontological Conceptualization
Akkerman, S. F., Bakker, A., & Penuel, W. R. (2021) · Educational Researcher · doi:10.3102/0013189x211028239 · Cited by 99
Professional learning networks: a conceptual model and research opportunities
Poortman, C. L., Brown, C., & Schildkamp, K. (2021) · Educational Research · doi:10.1080/00131881.2021.1985398 · Cited by 74
The Role of Context in Educational RCT Findings: A Call to Redefine “Evidence-Based Practice”
Kaplan, A., Cromley, J. G., Perez, T., Dai, T., Mara, K. R., & Balsai, M. J. (2020) · Educational Researcher · doi:10.3102/0013189x20921862 · Cited by 64
Spatial thinking as the missing piece in mathematics curricula
Gilligan-Lee, K. A., Hawes, Z. C. K., & Mix, K. S. (2022) · npj Science of Learning · doi:10.1038/s41539-022-00128-9 · Cited by 64
A valid evaluation of the theory of multiple intelligences is not yet possible: Problems of methodological quality for intervention studies
Ferrero, M., Vadillo, M. A., & León, S. P. (2021) · Intelligence · doi:10.1016/j.intell.2021.101566 · Cited by 98
Rethinking Social Action through Music: The Search for Coexistence and Citizenship in Medellín’s Music Schools
Baker, G. (2021) · Open Book Publishers · doi:10.11647/obp.0243
Growth-Mindset Intervention Delivered by Teachers Boosts Achievement in Early Adolescence
Porter, T., Molina, D. C., Cimpian, A., Roberts, S., Fredericks, A., Blackwell, L. S., & Trzesniewski, K. H. (2022) · Psychological Science · doi:10.1177/09567976211061109 · Cited by 75
A new methodological approach for evaluating the impact of educational intervention implementation on learning outcomes
Outhwaite, L. A., Gulliford, A., & Pitchford, N. J. (2019) · International Journal of Research & Method in Education · doi:10.1080/1743727x.2019.1657081 · Cited by 71
Neglect of publication bias compromises meta-analyses of educational research
Ropovik, I., Adamkovic, M., & Greger, D. (2021) · PLoS ONE · doi:10.1371/journal.pone.0252415 · Cited by 55
Working Out What Works: The Case of the Education Endowment Foundation in England
Edovald, T., & Nevill, C. (2020) · ECNU Review of Education · doi:10.1177/2096531120913039 · Cited by 49
The Effect-Size Benchmark That Matters Most: Education Interventions Often Fail
Kraft, M. A. (2023) · Educational Researcher · doi:10.3102/0013189x231155154 · Cited by 60
Making sense of effect sizes: Systematic differences in intervention effect sizes by outcome measure type
Wolf, B., & Harbatkin, E. (2023) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2071364 · Cited by 45
How malleable are cognitive abilities? A critical perspective on popular brief interventions.
Moreau, D. (2021) · American Psychologist · doi:10.1037/amp0000872 · Cited by 48
Home-based shared book reading interventions and children’s language skills: a meta-analysis of randomised controlled trials
Barone, C., Chambuleyron, E., Vonnak, R., & Assirelli, G. (2019) · Educational Research and Evaluation · doi:10.1080/13803611.2020.1814820 · Cited by 53
The dream of Sisyphus: Mathematics education in South Africa
Taylor, N. (2021) · South African Journal of Childhood Education · doi:10.4102/sajce.v11i1.911 · Cited by 42
A Commentary on Bowers (2020) and the Role of Phonics Instruction in Reading
Fletcher, J. M., Savage, R. S., & Vaughn, S. R. (2020) · Educational Psychology Review · doi:10.1007/s10648-020-09580-8 · Cited by 43
An evaluation of the impact of flipped‐classroom teaching on mathematics proficiency and self‐efficacy in Saudi Arabia
Algarni, B., & Lortie‐Forgues, H. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13250 · Cited by 56
Self-citation
Challenging the gold standard consensus: Randomised controlled trials (RCTs) and their pitfalls in evidence-based education
Parra, J. D., & Edwards, D. B. (2024) · Critical Studies in Education · doi:10.1080/17508487.2024.2314118 · Cited by 36
When Do Growth Mindset Interventions Work?
Miller, D. I. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.08.005 · Cited by 40
From replication to substantiation: A complexity theory perspective
Al‐Hoorie, A. H., Hiver, P. V., Larsen‐Freeman, D., & Lowie, W. M. (2021) · Language Teaching · doi:10.1017/s0261444821000409 · Cited by 39
Promoting learning from null or negative results in prevention science trials
Axford, N., Berry, V., Lloyd, J., Hobbs, T., & Wyatt, K. (2022) · Prevention Science · doi:10.1007/s11121-020-01140-4 · Cited by 34
Evidence-based education: Objections and future directions
Dekker, I., & Meeter, M. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.941410 · Cited by 57
Perceived quality of instruction: The relationship among indicators of students’ basic needs, mastery goals, and academic achievement
Theis, D., Sauerwein, M., & Fischer, N. (2019) · British Journal of Educational Psychology · doi:10.1111/bjep.12313 · Cited by 39
Experiment’s persistent failure in education inquiry, and why it keeps failing
Thomas, G. (2020) · British Educational Research Journal · doi:10.1002/berj.3660 · Cited by 39
Promoting a Growth Mindset in CS1: Does One Size Fit All? A Pilot Study
Quille, K., & Bergin, S. (2020) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3341525.3387361 · Cited by 30
Impact of programming on primary mathematics learning
Laurent-Chevalier, M., Crisci, R., Bressoux, P., Chaachoua, H., Nurra, C., de Vries, E., & Tchounikine, P. (2022) · Learning and Instruction · doi:10.1016/j.learninstruc.2022.101667 · Cited by 30
The Impact of Summer Programs on Student Mathematics Achievement: A Meta-Analysis
Lynch, K., An, L., & Mancenido, Z. (2022) · Review of Educational Research · doi:10.3102/00346543221105543 · Cited by 34
What can Implementation Science tell us about scaling interventions in school settings? A scoping review
Ryan, A., Prieto, E., Miller, A., & Gore, J. M. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100620 · Cited by 46
Efficacy of a School-Based Comprehensive Intervention Program for Adolescents With Autism
Hume, K., Odom, S. L., Steinbrenner, J. R. D., DaWalt, L. S., Hall, L. J., Kraemer, B., Tomaszewski, B. R., Brum, C., Szidon, K., & Bolt, D. M. (2021) · Exceptional Children · doi:10.1177/00144029211062589 · Cited by 26
Improving student outcomes through professional development: Protocol for a cluster randomised controlled trial of Quality Teaching Rounds
Miller, A., Gore, J. M., Wallington, C., Harris, J., Prieto, E., & Smith, M. (2019) · International Journal of Educational Research · doi:10.1016/j.ijer.2019.09.002 · Cited by 26
Reducing ‘avoidable research waste’ in applied linguistics research: Insights from healthcare research
Isaacs, T., & Chalmers, H. (2023) · Language Teaching · doi:10.1017/s0261444823000411 · Cited by 26
Quantifying “Promising Trials Bias” in Randomized Controlled Trials in Education
Sims, S., Anders, J., Inglis, M., & Lortie‐Forgues, H. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2090470 · Cited by 30
Self-citation
A critical review of recent economics pedagogy literature, 2020–2021
Birdi, A., Cook, S., Elliott, C., Lait, A., Mehari, T., & Wood, M. (2023) · International Review of Economics Education · doi:10.1016/j.iree.2023.100264 · Cited by 27
On the misinterpretation of effect size
Simpson, A. P. (2019) · Educational Studies in Mathematics · doi:10.1007/s10649-019-09924-4 · Cited by 22
Learning, Marginalization, and Improving the Quality of Education in Low-income Countries
Online, L. A. T. B. O. T. P. (. 2., Wagner, D. A., Castillo, N. M., & Lewis, S. G. (2022) · Open Book Publishers · doi:10.11647/obp.0256 · Cited by 38
5. Reducing Inequality in Education Using “Smaller, Quicker, Cheaper” Assessments
Crouch, L. A., & Slade, T. S. (2022) · Open Book Publishers · doi:10.11647/obp.0256.05 · Cited by 20
Effectiveness of a minecraft education intervention for improving spatial thinking in primary school children: A mixed methods two-level cluster randomised trial
Slattery, E. J., Butler, D., Marshall, K., Barrett, M. J., Hyland, N., O'Leary, M. R., & McAvinue, L. P. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.102003 · Cited by 35
A Glass Half Full and Half Empty: The State of the Science in Early Childhood Prevention and Intervention Research
Fisher, P. A., Barker, T. V., & Blaisdell, K. N. (2020) · Annual Review of Developmental Psychology · doi:10.1146/annurev-devpsych-121318-084958 · Cited by 23
A Path to Greater Credibility: Large-Scale Collaborative Education Research
Makel, M. C., Smith, K. N., McBee, M. T., Peters, S. J., & Miller, E. M. (2019) · AERA Open · doi:10.1177/2332858419891963 · Cited by 27
Education of Students with Disabilities, Science, and Randomized Controlled Trials
Odom, S. L. (2021) · Research and Practice for Persons with Severe Disabilities · doi:10.1177/15407969211032341 · Cited by 21
Groundhog day for inclusive education
Thomas, G., & Loxley, A. (2022) · Support for Learning · doi:10.1111/1467-9604.12406 · Cited by 19
Meta-Analyses and Narrative Review of Home-Based Interventions to Improve Literacy and Mathematics Outcomes for Children Between the Ages of 3 and 5 Years Old
Cahoon, A., Lira, C. J., Pérez, N. E., Pando, E. V. B., Campver, Y., García, D. S. P., & Simms, V. (2023) · Review of Educational Research · doi:10.3102/00346543231212491 · Cited by 25
Toward a System of Evidence for All: Current Practices and Future Opportunities in 37 Randomized Trials
Tipton, E., Spybrook, J. K., Fitzgerald, K. G., Wang, Q., & Davidson, C. (2020) · Educational Researcher · doi:10.3102/0013189x20960686 · Cited by 18
Reflective Goal-Setting Improves Academic Performance in Teacher and Business Education: A Large-Scale Field Experiment
Dekker, I., Schippers, M. C., & van Schooten, E. (2021) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2023.2231440 · Cited by 28
AIED: From Cognitive Simulations to Learning Engineering, with Humans in the Middle
McNamara, D. S. (2023) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00349-y · Cited by 18
Achieving Better Educational Practices Through Research Evidence: A Critical Analysis and Case Illustration of Benefits and Challenges
Ross, S. M., & Morrison, J. (2020) · ECNU Review of Education · doi:10.1177/2096531120916742 · Cited by 27
The case for mixed methods research: Embracing qualitative research to understand the (informal) economy
Cartwright, E. J., & Igudia, E. (2023) · Review of Development Economics · doi:10.1111/rode.13069 · Cited by 31
Complex adaptive interventions: The challenge ahead for instructed second language acquisition research
Hiver, P. V., & Nagle, C. (2024) · Annual Review of Applied Linguistics · doi:10.1017/s0267190524000060 · Cited by 17
Balancing Rigor, Replication, and Relevance: A Case for Multiple-Cohort, Longitudinal Experiments
Blazar, D., & Kraft, M. A. (2019) · AERA Open · doi:10.1177/2332858419876252 · Cited by 16
COVID derailed learning for 1.6 billion students. Here’s how schools can help them catch up
Pearson, H. (2022) · Nature · doi:10.1038/d41586-022-01387-7 · Cited by 16
The limits of inference: reassessing causality in international assessments
Rutkowski, D., Rutkowski, L. A., Thompson, G., & Canbolat, Y. (2024) · Large-scale Assessments in Education · doi:10.1186/s40536-024-00197-9 · Cited by 14
LURKING INFERENTIAL MONSTERS? QUANTIFYING SELECTION BIAS IN EVALUATIONS OF SCHOOL PROGRAMS
Weidmann, B., & Miratrix, L. W. (2020) · Journal of Policy Analysis and Management · doi:10.1002/pam.22236 · Cited by 13
Implementing Confidence Assessment in Low-Stakes, Formative Mathematics Assessments
Foster, C. F. (2021) · International Journal of Science and Mathematics Education · doi:10.1007/s10763-021-10207-9 · Cited by 20
Improving students’ financial literacy by training teachers using an online professional development module
Compen, B., Witte, K. D., Declercq, K., & Schelfhout, W. (2022) · Education Economics · doi:10.1080/09645292.2022.2035322 · Cited by 17
Challenges for implementation in diverse settings: reflections on two randomised controlled trials of educational interventions in South American communities
Newbury, D. F., Mesa, C., Puglisi, M. L., Nash, M., Nag, S., Hulme, C., & Snowling, M. J. (2022) · Research Papers in Education · doi:10.1080/02671522.2022.2065526 · Cited by 12
Fostering Engagement, Reflexivity, and 21st-Century Skills in Middle School: A Pilot Collaborative Action Research on Identity Formation with Adolescent Co-Researchers
Haag, P., Fantoni, T., & Dubal, S. (2022) · Journal of Intelligence · doi:10.3390/jintelligence10030064 · Cited by 12
The Returns to Education: A Meta‐Study
Clark, G., & Nielsen, C. A. A. (2026) · Kyklos · doi:10.1111/kykl.70041 · Cited by 12
Implementing Remote Developmental Research: A Case Study of a Randomized Controlled Trial Language Intervention During COVID-19
Ozernov‐Palchik, O., Olson, H. A., Arechiga, X. M., Kentala, H., Solorio-Fielder, J. L., Wang, K. L., Torres, Y. C., Gardino, N. D., Dieffenbach, J. R., & Gabrieli, J. D. E. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.734375 · Cited by 16
What counts as ‘evidence’ in literacy education?
Derewianka, B., Harper, H., Parkin, B., Acevedo, C., Rose, D., Dare, B., Brisk, M. E., & Jones, P. T. (2024) · The Australian Journal of Language and Literacy · doi:10.1007/s44020-024-00071-9 · Cited by 11
Standards in education: reforms, stagnation and the need to rethink
Bolden, D. S., & Tymms, P. B. (2020) · Oxford Review of Education · doi:10.1080/03054985.2020.1781608 · Cited by 18
Realizing the promises of replication studies in education
Morrison, K. (2019) · Educational Research and Evaluation · doi:10.1080/13803611.2020.1838300 · Cited by 11
An Integrative Qualitative Framework: Improving Research Through Strategic Mapping
Coker, D. C. (2023) · International Research in Education · doi:10.5296/ire.v11i1.20921 · Cited by 18
Evaluating What Works in the Classroom: Best Practice and Future Opportunities
Bowen, A. E. J., Palmer, L. R. J., Perry, R. C., Thomas, M. S. C., Tolmie, A. K., Borst, G., & Herwegen, J. V. (2024) · Mind Brain and Education · doi:10.1111/mbe.12430 · Cited by 10
Missing, Presumed different: Quantifying the risk of Attrition Bias in Education Evaluations
Weidmann, B., & Miratrix, L. W. (2021) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12677 · Cited by 10
Launching Registered Report Replications in Computer Science Education Research
Brown, N. C. C., Marinus, E., & Cheuoua, A. H. (2022) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3501385.3543971 · Cited by 10
Learning from failure: A context‐informed perspective on RCTs
Coldwell, M., & Moore, N. (2024) · British Educational Research Journal · doi:10.1002/berj.3960 · Cited by 13
Single- and Multilevel Perspectives on Covariate Selection in Randomized Intervention Studies on Student Achievement
Stallasch, S. E., Lüdtke, O., Artelt, C., Hedges, L. V., & Brunner, M. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09898-7 · Cited by 9
Latent value in humiliation: A design thinking tool to enhance empathy in creative ideation
Svihla, V., & Kachelmeier, L. (2021) · International Journal of Design Creativity and Innovation · doi:10.1080/21650349.2021.1976677 · Cited by 9
Research–practice partnerships in education: Lessons from the United States
France, R. (2025) · British Educational Research Journal · doi:10.1002/berj.4132 · Cited by 8
Replanteando la acción social por la música
Baker, G. (2022) · Open Book Publishers · doi:10.11647/obp.0263 · Cited by 8
Multiverses, Myths, and the Big Bang
Coker, D. C. (2022) · Advances in educational marketing, administration, and leadership book series · doi:10.4018/978-1-6684-4144-2.ch022 · Cited by 8
Evidence, Policy, Education, and Neuroscience—The State of Play in the UK
Thomas, M. S. C., Howard‐Jones, P. A., Dudman‐Jones, J., Palmer, L. R. J., Bowen, A. E. J., & Perry, R. C. (2024) · Mind Brain and Education · doi:10.1111/mbe.12423 · Cited by 9
Absolute effects of schooling as a reference for the interpretation of educational intervention effects
Luyten, H., Merrell, C. H., & Tymms, P. B. (2020) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2020.100939 · Cited by 8
Improving SEL outcomes for US 9th graders: Results from a District Wide School Reform
Rosen, J. A., Hong, Y., Rutledge, S. A., Cannata, M. A., Rotermund, S., & Leu, K. (2022) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2022.101212 · Cited by 8
Focusing on spontaneous focusing
Inglis, M. (2020) · Mathematical Thinking and Learning · doi:10.1080/10986065.2020.1818472 · Cited by 8
Self-citation
Open Science 2.0: Large-Scale Collaborative Education ResearchPreprint
Makel, M. C., Smith, K. N., McBee, M. T., Peters, S. J., & Miller, E. M. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/ypmjg · Cited by 9
Performance-based accountability systems at the organizational level: effects of a school program
Andersen, S. C., Hvidman, U., Junge, S. Y., & Rangvid, B. S. (2025) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muaf007 · Cited by 10
3. Teaching at the Bottom of the Pyramid
Akyeampong, K. (2022) · Open Book Publishers · doi:10.11647/obp.0256.03 · Cited by 8
4. Improving the Impact of Educational Technologies on Learning Within Low-Income Contexts
Castillo, N. M., Adam, T., & Haßler, B. (2022) · Open Book Publishers · doi:10.11647/obp.0256.04 · Cited by 7
A Recipe for Disappointment: Policy, Effect Size, and the Winner’s Curse
Simpson, A. P. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2066588 · Cited by 7
Connecting learning environments to learning: Two examples from children’s mathematics
Siegler, R. S. (2024) · Developmental Review · doi:10.1016/j.dr.2024.101138 · Cited by 7
The Internal Validity of the School-Level Comparative Interrupted Time Series Design: Evidence From Four New Within-Study Comparisons
Sims, S., Anders, J., & Zieger, L. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2051652 · Cited by 7
Examining mathematics teacher motivation during lesson study: the role of contextual factors for perceived relatedness
Petty, C. S., Eddy, C. M., & Pratt, S. S. (2022) · Learning Environments Research · doi:10.1007/s10984-022-09421-1 · Cited by 11
I find this task interesting, so do you? Preservice teachers’ judgments of students’ enjoyment, boredom, and situational interest regarding tasks with and without a connection to reality
Kanefke, J., & Schukajlow, S. (2023) · Journal of Mathematics Teacher Education · doi:10.1007/s10857-023-09581-8 · Cited by 11
Education Technology
Sabatini, J., Park, J. H., Chakraborty, A., Graesser, A. C., Macintyre, T., Chandrasekharan, M., & Ogunniran, M. O. (2022) · UNESCO MGIEP eBooks · doi:10.56383/sdro8173 · Cited by 9
13. Ivory Coast
Jasińska, K. K., & Guei, S. (2022) · Open Book Publishers · doi:10.11647/obp.0256.13 · Cited by 7
How can we optimise learning from trials in child and adolescent mental health?
Axford, N., Berry, V., Lloyd, J., & Wyatt, K. M. (2022) · Evidence-Based Mental Health · doi:10.1136/ebmental-2022-300500 · Cited by 8
Reading Aloud to Children, Social Inequalities and Vocabulary Development: Evidence from a Randomized Controlled Trial
Barone, C., Fougère, D., & Martel, K. (2024) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2023.2283475 · Cited by 5
Exploring randomised controlled trials in education: key features and selected examples from Ireland
Slattery, E. J., Clerkin, A., Gilleece, L., Butler, D., Marshall, K., & Hedges, L. V. (2025) · Irish Educational Studies · doi:10.1080/03323315.2025.2523301 · Cited by 6

Authors: Lortie-Forgues, H., & Inglis, M.
Affiliations: University of York, UK
Loughborough University (GB)
ORCID: Inglis, Matthew
Venue: Educational Researcher
Publisher: American Educational Research Association (AERA)
Published: 2019-03-11
Volume: 48
Issue: 3
Pages: 158-166
ISSN: 0013-189X (ISSN-L)
1935-102X
DOI: 10.3102/0013189x19832850
OpenAlex: W2919428740
Cited by: 234
References: 44
Citations by year: 2026: 19 · 2025: 20 · 2024: 27 · 2023: 19 · 2022: 64 · 2021: 39 · 2020: 30 · 2019: 16
FWCI: 24.93
Type: journal article
Language: en
Open access: green
orcid.org
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Educational Researcher (publishedVersion)
White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
Loughborough University Institutional Repository (Loughborough University) (submittedVersion, cc-by-nc-nd)
Loughborough University Research Repository (Loughborough University) (submittedVersion, cc-by-nc-nd)
Funders: university of york
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Education › School Choice and Performance
Social Sciences › Education › Educational Research and Methods
Keywords: educational intervention, effect size, confidence intervals, Bayes factor, statistical power, Education Endowment Foundation, evidence-based education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Rigorous Large-Scale Educational RCTs Are Often Uninformative: Should We Be Concerned?

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks