The DOI matches its Crossref record; 120 of the 123 references it cites were found in a registry or an index.
Strengthening the Research Base That Informs STEM Instructional Improvement Efforts: A Meta-Analysis
Abstract
We present results from a meta-analysis of 95 experimental and quasi-experimental pre-K–12 science, technology, engineering, and mathematics (STEM) professional development and curriculum programs, seeking to understand what content, activities, and formats relate to stronger student outcomes. Across rigorously conducted studies, we found an average weighted impact estimate of +0.21 standard deviations. Programs saw stronger outcomes when they helped teachers learn to use curriculum materials; focused on improving teachers’ content knowledge, pedagogical content knowledge, and/or understanding of how students learn; incorporated summer workshops; and included teacher meetings to troubleshoot and discuss classroom implementation. We discuss implications for policy and practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Similarly, Lynch et al. (2019) found no significant relation between these variables.”
From Teacher Professional Development and Student Mathematics Achievement: A Meta-Analysis of the Effects and Moderators (2025) · cites this paper as Lynch et al. (2019)
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
Those Who Understand: Knowledge Growth in Teaching
SHULMAN, L. S. (1986) · Educational Researcher · doi:10.3102/0013189x015002004 · Cited by 18,546
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
(2013) · Cited by 6,708
Desimone, L. M. (2009) · Educational Researcher · doi:10.3102/0013189x08331140 · Cited by 4,862
Second Handbook of Research on Mathematics Teaching and Learning.
Lester, F. (2007) · Cited by 3,090
Scientific Research in Education
(2002) · National Academies Press eBooks · doi:10.17226/10236 · Cited by 1,723
Research works printed but not linked: 28
- Agodini R., 2013, After two years, three elementary math curricula outperform a fourth
- 10.1037/0022-0663.94.4.762 · doi:10.1037/0022-0663.94.4.762 · Paper page
- 10.3102/0013189x025009006 · doi:10.3102/0013189x025009006 · Paper page
- Becker K., 2011, Journal of STEM Education: Innovations and Research, 12, 23
- Blank R. K., 2010, Paper presented at the Society for Research on Educational Effectiveness Spring Conference
- Borman K. M., 2009, Paper presented at the Society for Research on Educational Effectiveness Spring Conference
- Chambers E. A. (2003). Efficacy of educational technology in elementary and secondary classrooms: A meta-analysis of the research literature from 1992–2002 (Unpublished doctoral dissertation). Southern Illinois University Carbondale.
- Darling-Hammond L., 1995, Phi Delta Kappan, 76, 597
- Devlin-Scherer W., 1998, Paper presented at the Annual Meeting of the American Association of Colleges for Teacher Education
- Fisher Z., Tipton E. (2015). robumeta: An R-package for robust variance estimation in meta-analysis. Retrieved from https://arxiv.org/pdf/1503.02220.pdf · arXiv
- Frechtling J., 1995, Professional development programs: A perspective on the last four decades
- Garet M. S., 2010, Middle school mathematics professional development impact study: Findings after the first year of implementation
- 10.1037/0033-2909.88.2.359 · DOI · Paper page
- Kennedy M. M., 1999, Form and substance in in-service teacher education
- Kim J. S., 2012, Paper presented at the annual meeting of the Society for Research on Educational Effectiveness
- Kulik J. A., 2003, Effects of using instructional technology in elementary and secondary schools: What controlled evaluation studies say
- Miles K. H., 2004, Journal of Education Finance, 30, 1
- Miller B., 1994, Staff development for teachers: A study of configurations and costs in four districts
- Murphy R., 2002, E-DESK: A review of recent evidence on discrete educational software
- National Research Council, 2004, On evaluating curricular effectiveness: Judging the quality of K-12 mathematics evaluations
- National Research Council, 2011, Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics
- Presser A. L., 2015, Paper presented at the Society for Research on Educational Effectiveness Spring Conference
- Resendez M., 2006, 2005 Prentice Hall Science Explorer randomized control trial
- Russell S. J., 2006, Investigations in Number, Data, and Space. Grade 1
- Timperley H., 2008, Teacher professional learning and development
- Van Egeren L. A., 2014, Poster presented at the 2014 Discovery Research K-12 PI Meeting
- What Works Clearinghouse, 2008, Middle school math curricula
- What Works Clearinghouse, 2010, Procedures and standards handbook
Cited by
Interpreting effect sizes of education interventions
Kraft, M. A. (2020) · Educational Researcher · doi:10.3102/0013189x20912798 · Cited by 1,281
“…among measures of student achievement, effect sizes for researcher-designed and specialized topic tests aligned with the treatment are often two to four times larger than effects on broad standardized state tests (Cheung & Slavin, 2016; Hill et al., 2008; Lipsey et al., 2012; Lynch et al., 2019).”
Identifying the characteristics of effective teacher professional development: a critical review
Sims, S., & Fletcher-Wood, H. (2021) · School Effectiveness and School Improvement · doi:10.1080/09243453.2020.1772841 · Cited by 582
“In particular, this approach identified a number of reviews which did not endorse the consensus view (e.g. Kennedy, 2016; Lynch et al., 2019; Yoon, 2011; Kraft, Blazar, & Hogan, 2018), as well as a wide range of relevant original research papers.”
BackgroundResultSablić, M., Mirosavljević, A., & Škugor, A. (2021) · Technology, Knowledge and Learning · doi:10.1007/s10758-020-09455-5 · Cited by 235
“Watching the video, shaping lessons, studying students’ work and similar professional development activities may differentially affect program outcomes such as curriculum design features, e.g. length of implementation and degree of teacher support (Lynch et al. 2019).”
BackgroundEffective Teacher Professional Development: New Theory and a Meta-Analytic Test
Sims, S., Fletcher-Wood, H., O’Mara-Eves, A., Cottingham, S., Stansfield, C., Goodrich, J., Van Herwegen, J., & Anders, J. (2025) · Review of Educational Research · doi:10.3102/00346543231217480 · Cited by 168
“A second strand of research has used meta-regression to investigate the associations between different aspects of PD design and impact on pupil outcomes (Basma & Savage, 2017; Didion et al., 2020; Kraft et al., 2018; Lynch et al, 2019).”
Huang, B., Siu-Yung Jong, M., Tu, Y.-F., Hwang, G.-J., Chai, C. S., & Yi-Chao Jiang, M. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104577 · Cited by 167
Metadata
- Authors
- Lynch, K., Hill, H. C., Gonzalez, K. E., & Pollard, C.
- Affiliations
- Brown University (US)
- Harvard University (US)
- Venue
- Educational Evaluation and Policy Analysis
- Published
- 2019-06-03
- Volume
- 41
- Issue
- 3
- Pages
- 260-293
- ISSN
- 0162-3737 (ISSN-L)
- 1935-1062
- OpenAlex
- W2948530858
- Cited by
- 160
- References
- 123
- Citations by year
- 2026: 27 · 2025: 38 · 2024: 21 · 2023: 24 · 2022: 21 · 2021: 17 · 2020: 9 · 2019: 3
- FWCI
- 3.58
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Educational Evaluation and Policy Analysis (publishedVersion)
- Funders
- Directorate for Education and Human Resources (1348669)
- Topics
- Social Sciences › Education › Mathematics Education and Teaching Techniques
- Social Sciences › Education › Science Education and Pedagogy
- Social Sciences › Education › Teacher Education and Leadership Studies
- Keywords
- STEM education, teacher professional development, curriculum implementation, pedagogical content knowledge, teacher content knowledge, summer workshop, teacher collaboration, student outcomes, teaching improvement, K-12 education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Strengthening the Research Base That Informs STEM Instructional Improvement Efforts: A Meta-Analysis. https://app.cochat.ai/papers/10.3102/0162373719849044
- The paper
- Lynch, K., Hill, H. C., Gonzalez, K. E., & Pollard, C. (2019). Strengthening the Research Base That Informs STEM Instructional Improvement Efforts: A Meta-Analysis. Educational Evaluation and Policy Analysis. https://doi.org/10.3102/0162373719849044
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.