The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Use of Quasi-Experimental Research Designs in Education Research: Growth, Promise, and Challenges
Abstract
In the past few decades, we have seen a rapid proliferation in the use of quasi-experimental research designs in education research. This trend, stemming in part from the “credibility revolution” in the social sciences, particularly economics, is notable along with the increasing use of randomized controlled trials in the strive toward rigorous causal inference. The overarching purpose of this chapter is to explore and document the growth, applicability, promise, and limitations of quasi-experimental research designs in education research. We first provide an overview of widely used quasi-experimental research methods in this growing literature, with particular emphasis on articles from the top ranked education research journals, including those published by the American Educational Research Association. Next, we demonstrate the applicability and promise of these methods in enhancing our understanding of the causal effects of education policies and interventions using key examples and case studies culled from the extant literature across the pre-K–16 education spectrum. Finally, we explore the limitations of these methods and conclude with thoughts on how education researchers can adapt these innovative, interdisciplinary techniques to further our understanding of some of the most enduring questions in educational policy and practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Matching Methods for Causal Inference: A Review and a Look Forward
Stuart, E. A. (2010) · Statistical Science · doi:10.1214/09-sts313 · Cited by 5,478
Skill formation and the economics of investing in disadvantaged children
Heckman, J. J. (2006) · Science · doi:10.1126/science.1128898 · Cited by 3,559
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841 · cited 4×
“With the proliferation of the use of QEDs across the social sciences, scholars are also raising concerns regarding the trade-offs inherent in the overreliance on the use of these methods for research and practice more broadly (Deaton & Cartwright, 2018; Pritchett, 2018).”
BackgroundMethodInfluentialAssessing the Effects of School Resources on Student Performance: An Update
Hanushek, E. A. (1997) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737019002141 · Cited by 1,612
Angrist, J. D., & Pischke, J.-S. (2010) · Journal of Economic Perspectives · doi:10.1257/jep.24.2.3 · Cited by 1,566
Designing Difference in Difference Studies: Best Practices for Public Health Policy Research
Wing, C., Simon, K., & Bello-Gomez, R. A. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-013507 · Cited by 1,470 · cited 1×
“For example, the parallel trends assumption inherent in DID designs are central to the validity of the estimated effects (see Wing et al., 2018, for a review of the robustness checks necessary to rigorously defend these assumptions).”
BackgroundRegression-discontinuity analysis: An alternative to the ex post facto experiment.
Thistlethwaite, D. L., & Campbell, D. T. (1960) · Journal of Educational Psychology · doi:10.1037/h0044319 · Cited by 1,469
The Failure of Input‐based Schooling Policies
Hanushek, E. A. (2003) · The Economic Journal · doi:10.1111/1468-0297.00099 · Cited by 1,164
Factors relevant to the validity of experiments in social settings.
Campbell, D. T. (1957) · Psychological Bulletin · doi:10.1037/h0040950 · Cited by 1,140
Why high-order polynomials should not be used in regression discontinuity designs
Gelman, A., & Imbens, G. (2019) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2017.1366909 · Cited by 1,101
Longer-Term Effects of Head Start
Garces, E., Thomas, D., & Currie, J. (2002) · American Economic Review · doi:10.1257/00028280260344560 · Cited by 951
Hedges, L. V., Laine, R. D., & Greenwald, R. (1994) · Educational Researcher · doi:10.3102/0013189x023003005 · Cited by 819
Jackson, C. K., Johnson, R. C., & Persico, C. (2016) · The Quarterly Journal of Economics · doi:10.1093/qje/qjv036 · Cited by 805
Investing in Preschool Programs
Duncan, G. J., & Magnuson, K. (2013) · Journal of Economic Perspectives · doi:10.1257/jep.27.2.109 · Cited by 762
Difference-in-Differences with Variation in Treatment Timing
Goodman-Bacon, A. (2018) · doi:10.3386/w25018 · Cited by 757
The Effects of Universal Pre-K on Cognitive Development.
Gormley, W. T., Gayer, T., Phillips, D., & Dawson, B. (2005) · Developmental Psychology · doi:10.1037/0012-1649.41.6.872 · Cited by 733
International encyclopedia of education
Peterson, P. L. (2010) · Cited by 728
Weiland, C., & Yoshikawa, H. (2013) · Child Development · doi:10.1111/cdev.12099 · Cited by 701
Early Childhood Intervention and Life-Cycle Skill Development: Evidence from Head Start
Deming, D. (2009) · American Economic Journal: Applied Economics · doi:10.1257/app.1.3.111 · Cited by 695 · cited 2×
“Studies that use family-fixed effects find that participation in ECE programs improves college attendance and completion, improves health, and reduces criminal justice involvement (Carneiro & Ginja, 2014; Deming, 2009; Garces et al., 2002).”
BackgroundMethodDoes prekindergarten improve school preparation and performance?
Magnuson, K. A., Ruhm, C., & Waldfogel, J. (2007) · Economics of Education Review · doi:10.1016/j.econedurev.2005.09.008 · Cited by 673
The Value of School Facility Investments: Evidence from a Dynamic Regression Discontinuity Design*
Cellini, S. R., Ferreira, F., & Rothstein, J. (2010) · Quarterly Journal of Economics · doi:10.1162/qjec.2010.125.1.215 · Cited by 532
School Finance Reform and the Distribution of Student Achievement
Lafortune, J., Rothstein, J., & Schanzenbach, D. W. (2018) · American Economic Journal: Applied Economics · doi:10.1257/app.20160567 · Cited by 415
Cook, T. D. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.002 · Cited by 303
Impacts of Early Childhood Education on Medium- and Long-Term Educational Outcomes
McCoy, D. C., Yoshikawa, H., Ziol-Guest, K. M., Duncan, G. J., Schindler, H. S., Magnuson, K., Yang, R., Koepp, A., & Shonkoff, J. P. (2017) · Educational Researcher · doi:10.3102/0013189x17737739 · Cited by 284
The Behavioral and Distributional Implications of Aid for College
Dynarski, S. (2002) · American Economic Review · doi:10.1257/000282802320189401 · Cited by 281 · cited 1×
“…to students above some academic eligibility threshold, such as Georgia’s HOPE scholarship program that historically covered a portion of college costs for students with a 3.0 high school grade point average (GPA) or higher (see Dynarski, 2002, for more detailed discussion of the HOPE scholarship).”
BackgroundInfluentialvan Huizen, T., & Plantenga, J. (2018) · Economics of Education Review · doi:10.1016/j.econedurev.2018.08.001 · Cited by 243
Long-Term Impacts of Compensatory Preschool on Health and Behavior: Evidence from Head Start
Carneiro, P., & Ginja, R. (2014) · American Economic Journal: Economic Policy · doi:10.1257/pol.6.4.135 · Cited by 215
Replication and robustness in developmental research
Duncan, G. J., Engel, M., Claessens, A., & Dowsett, C. J. (2014) · Developmental Psychology · doi:10.1037/a0037996 · Cited by 207
Performance Funding in Higher Education: Do Financial Incentives Impact College Completions?
Hillman, N., Tandberg, D. A., & Gross, J. P. K. (2014) · The Journal of Higher Education · doi:10.1353/jhe.2014.0031 · Cited by 199
Evaluating the Impacts of “New” Performance Funding in Higher Education
Hillman, N. W., Tandberg, D. A., & Fryar, A. H. (2015) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373714560224 · Cited by 182
Johnson, R. C., & Jackson, C. K. (2019) · American Economic Journal: Economic Policy · doi:10.1257/pol.20180510 · Cited by 178
State Higher Education Performance Funding: Data, Outcomes, and Policy Implications*
Tandberg, D. A., & Hillman, N. (2014) · Journal of education finance · doi:10.1353/jef.2014.a539805 · Cited by 175
Performance Funding for Higher Education
Dougherty, K. (2016) · Johns Hopkins University Press eBooks · doi:10.1353/book.47910 · Cited by 170
Evaluating Impacts of Performance Funding Policies on Student Outcomes in Higher Education
Rutherford, A., & Rabovsky, T. (2014) · The Annals of the American Academy of Political and Social Science · doi:10.1177/0002716214541048 · Cited by 165
An Examination of the (Un)Intended Consequences of Performance Funding in Higher Education
Umbricht, M. R., Fernandez, F., & Ortagus, J. C. (2015) · Educational Policy · doi:10.1177/0895904815614398 · Cited by 151
How Generalizable Is Your Experiment? An Index for Comparing Experimental Samples and Populations
Tipton, E. (2014) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998614558486 · Cited by 141
Evidence-Based Practices in a Changing World: Reconsidering the Counterfactual in Education Research
Lemons, C. J., Fuchs, D., Gilbert, J. K., & Fuchs, L. S. (2014) · Educational Researcher · doi:10.3102/0013189x14539189 · Cited by 141
Court-Ordered Finance Reforms in the Adequacy Era: Heterogeneous Causal Effects and Sensitivity
Candelaria, C., & Shores, K. (2017) · Education Finance and Policy · doi:10.1162/edfp_a_00236 · Cited by 133
Does School Spending Matter? The New Literature on an Old Question
Jackson, C. K. (2018) · National Bureau of Economic Research · doi:10.3386/w25368 · Cited by 117
Investing in schools: capital spending, facility conditions, and student achievement
Martorell, P., Stange, K., & McFarlin, I. (2016) · Journal of Public Economics · doi:10.1016/j.jpubeco.2016.05.002 · Cited by 116
A Review of Statistical Methods for Generalizing From Evaluations of Educational Interventions
Tipton, E., & Olsen, R. B. (2018) · Educational Researcher · doi:10.3102/0013189x18781522 · Cited by 114
Evaluating the Impact of Performance Funding in Ohio and Tennessee
Hillman, N., Fryar, A. H., & Crespín‐Trujillo, V. (2017) · American Educational Research Journal · doi:10.3102/0002831217732951 · Cited by 112
Gándara, D., & Rutherford, A. (2017) · Research in Higher Education · doi:10.1007/s11162-017-9483-x · Cited by 99
Barnett, W. S., Jung, K., Friedman‐Krauss, A. H., Frede, E., Nores, M., Hustedt, J. T., Howes, C. H., & Daniel-Echols, M. (2018) · AERA Open · doi:10.1177/2332858418766291 · Cited by 94
Head Start at Ages 3 and 4 Versus Head Start Followed by State Pre-K: Which Is More Effective?
Jenkins, J. M., Farkas, G., Duncan, G. J., Burchinal, M., & Vandell, D. L. (2015) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373715587965 · Cited by 93
Do Performance-Based Funding Policies Affect Underrepresented Student Enrollment?
Kelchen, R. (2018) · The Journal of Higher Education · doi:10.1080/00221546.2018.1434282 · Cited by 84
Differential Pricing in Undergraduate Education: Effects on Degree Production by Field
Stange, K. (2014) · Journal of Policy Analysis and Management · doi:10.1002/pam.21803 · Cited by 79
Morris, P., Connors, M. C., Friedman‐Krauss, A. H., McCoy, D. C., Weiland, C., Feller, A., Page, L. C., Bloom, H. S., & Yoshikawa, H. (2018) · AERA Open · doi:10.1177/2332858418769287 · Cited by 79
Tandberg, D. A., Hillman, N., & Barakat, M. (2014) · Teachers College Record The Voice of Scholarship in Education · doi:10.1177/016146811411601205 · Cited by 75
Does Performance-Based Funding Affect Colleges’ Financial Priorities?
Kelchen, R., & Stedrak, L. J. (2016) · Journal of education finance · doi:10.1353/jef.2016.0006 · Cited by 71
Challenges in Building Usable Knowledge in Education
Hedges, L. V. (2017) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2017.1375583 · Cited by 71
Schudde, L. T. (2018) · Review of Research in Education · doi:10.3102/0091732x18759040 · Cited by 67
Case selection and causal inferences in qualitative comparative research
Plümper, T., Troeger, V. E., & Neumayer, E. (2019) · PLoS ONE · doi:10.1371/journal.pone.0219727 · Cited by 66
Li, A. Y., & Kennedy, A. (2017) · Community College Review · doi:10.1177/0091552117743790 · Cited by 65
Head Start’s Long-Run Impact: Evidence from the Program’s Introduction
Thompson, O. (2017) · The Journal of Human Resources · doi:10.3368/jhr.53.4.0216-7735r1 · Cited by 58
Charter Schools and the Achievement Gap
Cohodes, S. (2018) · The Future of Children · doi:10.1353/foc.2018.0008 · Cited by 49
Characteristics of School Districts That Participate in Rigorous National Educational Evaluations
Stuart, E. A., Bell, S. H., Ebnesajjad, C. F., Olsen, R. B., & Orr, L. L. (2016) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2016.1205160 · Cited by 48
Lipsey, M. W., Weiland, C., Yoshikawa, H., Wilson, S. J., & Hofer, K. G. (2014) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373714547266 · Cited by 46
Limitations of Experiments in Education Research
Schanzenbach, D. W. (2012) · Education Finance and Policy · doi:10.1162/edfp_a_00063 · Cited by 45
Reback, R. (2010) · Education Finance and Policy · doi:10.1162/edfp.2010.5.2.5201 · Cited by 42
Who Benefits From Kindergarten? Evidence From the Introduction of State Subsidization
Dhuey, E. (2011) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373711398125 · Cited by 41
How to Create and Use State-Level Policy Data Sets in Education Research
Kelchen, R., Rosinger, K. O., & Ortagus, J. C. (2019) · AERA Open · doi:10.1177/2332858419873619 · Cited by 38
Self-citationShackling the Identification Police?
Ruhm, C. J. (2018) · National Bureau of Economic Research · doi:10.3386/w25320 · Cited by 38
Sources of Funding for Schools
Howell, P. L., & Miller, B. B. (1997) · The Future of Children · doi:10.2307/1602444 · Cited by 35
Financial Aid and First-Year Collegiate GPA: A Regression Discontinuity Approach
Curs, B. R., & Harper, C. E. (2012) · Review of higher education/The review of higher education · doi:10.1353/rhe.2012.0040 · Cited by 30
Kelchen, R. (2019) · Community College Review · doi:10.1177/0091552119865611 · Cited by 25
Regression Discontinuity and Beyond
Wing, C., & Bello-Gómez, R. A. (2018) · American Journal of Evaluation · doi:10.1177/1098214017736155 · Cited by 22
Wong, V. C., Wing, C., Martin, D., & Krishnamachari, A. (2017) · Educational Researcher · doi:10.3102/0013189x17743230 · Cited by 13
Research works printed but not linked: 7
- Callahan M. K., 2017, Impact of OBF on student outcomes: Tennessee and Indiana
- Currie J., Thomas D. (1995). Does Head Start make a difference? American Economic Review, 85(3), 341–364. https://www.jstor.org/stable/2118178
- Friedman-Krauss A. H., 2018, The state of preschool 2017
- Loeb S., 2017, Descriptive analysis in education: A guide for researchers, 2017
- National Research Council & Institute of Medicine Committee on Integrating the Science of Early Childhood Development. (2000). From neurons to neighborhoods: The science of early childhood development (Shonkoff J. P., Phillips D. A., Eds.). National Academies Press. http://www.ncbi.nlm.nih.gov/books/NBK225557/
- Panhans M. T., 2017, History of Political Economy, 49, 127
- Singer J. D. (2018). Even more challenges in building usable knowledge in education. Journal of Research on Educational Effectiveness, 11(1), 22–24. https://doi.org/10.1080/19345747.2017.1402397 · doi:10.1080/19345747.2017.1402397
Cited by
Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study
Mahapatra, S. (2024) · Smart Learning Environments · doi:10.1186/s40561-024-00295-9 · Cited by 398
Muirhead, K., Macaden, L., Smyth, K., Chandler, C., Clarke, C., Polson, R., & O’Malley, C. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01781-8 · Cited by 32
“Quasiexperimental studies can also aid in understanding causal effects; however, the reliability of causal claims and estimates varies across these designs and depends on how close the study conditions are to an experiment [39].”
BackgroundMethodological Choices When Assessing Summer Bridge Programs in STEM Majors: A Scoping Review
Caballero Díaz, D., Amin, A., Musa, P., & Leung, V. (2026) · Education Sciences · doi:10.3390/educsci16020220
Metadata
- Authors
- Gopalan, M., Rosinger, K., & Ahn, J. B.
- Venue
- Review of Research in Education
- Published
- 2020-03
- Volume
- 44
- Issue
- 1
- Pages
- 218-243
- ISSN
- 0091-732X (ISSN-L)
- 1935-1038
- OpenAlex
- W3018756979
- Cited by
- 411
- References
- 69
- FWCI
- 27.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.