Revisiting The Widget Effect : Teacher Evaluation Reforms and the Distribution of Teacher Effectiveness
The DOI matches its Crossref record; 37 of the 38 references it cites were found in a registry or an index.
Abstract
In 2009, the New Teacher Project’s The Widget Effect documented the failure of U.S. public school districts to recognize and act on differences in teacher effectiveness. We revisit these findings by compiling teacher performance ratings across 24 states that adopted major reforms to their teacher evaluation systems. In the vast majority of these states, the percentage of teachers rated unsatisfactory remains less than 1%. However, the full distributions of ratings vary widely across states, with 0.7% to 28.7% rated below proficient and 6% to 62% rated above proficient. We present original survey data from an urban district illustrating that evaluators perceive more than 3 times as many teachers in their schools to be below proficient than they rate as such. Interviews with principals reveal several potential explanations for these patterns.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Qualitative Data Analysis: An Expanded Sourcebook
(1994) · Journal of Environmental Psychology · doi:10.1016/s0272-4944(05)80231-2 · Cited by 48,420
Basics of Qualitative Research: Grounded Theory Procedures and Techniques.
Ellis, C., Strauss, A., & Corbin, J. (1992) · Contemporary Sociology · doi:10.2307/2074814 · Cited by 29,249
Basics of Qualitative Research (3rd ed.): Techniques and Procedures for Developing Grounded Theory
Corbin, J., & Strauss, A. (2008) · SAGE · doi:10.4135/9781452230153 · Cited by 12,136
Teachers, Schools, and Academic Achievement
Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005) · Econometrica · doi:10.1111/j.1468-0262.2005.00584.x · Cited by 4,501
(2010) · Encyclopedia of Curriculum Studies · doi:10.4135/9781412958806.n175 · Cited by 2,556
The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data
Rockoff, J. E. (2004) · American Economic Review · doi:10.1257/0002828041302244 · Cited by 2,432
(2002) · Cited by 2,212
Street-Level Bureaucracy: The Dilemmas of the Individual in Public Service
Lipsky, M. (1980) · Project Muse (Johns Hopkins University) · Cited by 2,135
(2009) · Issue Lab (Candid) · Cited by 552
Jacob, B. A., & Lefgren, L. (2008) · Journal of Labor Economics · doi:10.1086/522974 · Cited by 494
Research works printed but not linked: 5
- Donaldson M. L., 2009, So long, Lake Wobegon? Using teacher evaluation to raise teacher quality
- Glander M., 2015, Selected statistics from the public elementary and secondary education universe: School year 2013–14. First look
- Thomas E., 2010, Newsweek, 155, 24
- Toch T., 2008, Rush to judgment: Teacher evaluation in public education
- Wechsler M., 2007, The status of the teaching profession 2007
Cited by
Papay, J. P., Taylor, E. S., Tyler, J. H., & Laski, M. E. (2020) · American Economic Journal: Economic Policy · doi:10.1257/pol.20170709 · Cited by 148
Papay, J., Taylor, E., Tyler, J., & Laski, M. (2016) · doi:10.3386/w21986 · Cited by 148
Grissom, J. A., & Bartanen, B. (2018) · American Educational Research Journal · doi:10.3102/0002831218797931 · Cited by 229
Working and Learning Together: Rethinking Human Resource Policies for Schools
OECD (2019) · OECD reviews of school resources · doi:10.1787/b7aaf050-en · Cited by 203
What difference does school leadership make to student outcomes?
Robinson, V., & Gray, E. (2019) · Journal of the Royal Society of New Zealand · doi:10.1080/03036758.2019.1582075 · Cited by 109
“A recent evaluation of this reform showed that, despite substantial investment in a suite of validated evaluation tools, the proportion of teachers receiving a less than satisfactory rating changed little (Kraft and Gilmour 2017).”
BackgroundInfluential
Metadata
- Authors
- Kraft, M. A., & Gilmour, A. F.
- Affiliations
- Brown University (US)
- Temple University (US)
- Venue
- Educational Researcher
- Published
- 2017-06
- Volume
- 46
- Issue
- 5
- Pages
- 234-249
- ISSN
- 0013-189X (ISSN-L)
- 1935-102X
- OpenAlex
- W2739034820
- Cited by
- 211
- References
- 38
- Citations by year
- 2026: 19 · 2025: 16 · 2024: 24 · 2023: 23 · 2022: 19 · 2021: 24 · 2020: 29 · 2019: 29 · 2018: 22 · 2017: 4
- FWCI
- 65.27
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Educational Researcher (publishedVersion)
- Topics
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Education › School Leadership and Teacher Performance
- Social Sciences › Education › Teacher Education and Assessments
- Keywords
- teacher evaluation reform, teacher effectiveness, teacher performance ratings, rating distribution, evaluator perceptions, principal interview, urban school districts
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Revisiting The Widget Effect : Teacher Evaluation Reforms and the Distribution of Teacher Effectiveness. https://app.cochat.ai/papers/10.3102/0013189x17718797
- The paper
- Kraft, M. A., & Gilmour, A. F. (2017). Revisiting The Widget Effect : Teacher Evaluation Reforms and the Distribution of Teacher Effectiveness. Educational Researcher. https://doi.org/10.3102/0013189x17718797
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.