Observational Evaluation of Teachers: Measuring More Than We Bargained for?
The DOI matches its Crossref record; 23 of the 24 references it cites were found in a registry or an index.
Abstract
Our secondary analysis of Measures of Effective Teaching data contributes to growing evidence that observation ratings, used as part of comprehensive teacher evaluation systems across the nation, may measure factors outside of a teacher’s performance or control. Specifically, men and teachers in classrooms with high concentrations of Black, Hispanic, male, and low-performing students receive significantly lower observation ratings. By using various methodological approaches and a subsample of teachers randomly assigned to classrooms, we demonstrate that these differences are unlikely due to actual differences in teacher quality. These results suggest that policymakers consider the unintended consequences of using observational ratings to evaluate teachers and consider ways to adjust ratings to ensure they are fair.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Teacher Sorting and the Plight of Urban Schools: A Descriptive Analysis
Lankford, H., Loeb, S., & Wyckoff, J. (2002) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737024001037 · Cited by 1,454
How Teaching Conditions Predict Teacher Turnover in California Schools
Loeb, S., Darling-Hammond, L., & Luczak, J. (2005) · Peabody Journal of Education · doi:10.1207/s15327930pje8003_4 · Cited by 610
Facts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609 · cited 1×
“In advocating for more replication studies in educational research, for example, Makel and Plucker (2014) revealed that less than 1% of education articles in top journals were replication studies.”
An Interaction-Based Approach to Enhancing Secondary School Instruction and Student Achievement
Allen, J. P., Pianta, R. C., Gregory, A., Mikami, A. Y., & Lun, J. (2011) · Science · doi:10.1126/science.1207998 · Cited by 590
Kraft, M. A., & Papay, J. P. (2014) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373713519496 · Cited by 517 · cited 2×
“…in schools with large proportions of marginalized students due to worse working conditions of these schools, including worse collaboration and leadership quality, known to influence teaching quality and retention (Kraft & Papay, 2014; Loeb, Darling-Hammond, & Luczak, 2005; Ronfeldt, 2015).”
BackgroundOn Making Determinations of Quality in Teaching
Fenstermacher, G. D., & Richardson, V. (2005) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1111/j.1467-9620.2005.00462.x · Cited by 497
Papay, J. P., & Kraft, M. A. (2015) · Journal of Public Economics · doi:10.1016/j.jpubeco.2015.02.008 · Cited by 370 · cited 6×
“A recent study by Steinberg and Garrett (2016) provides the strongest experimental estimates to date that students’ incoming performance indeed predicts better observation ratings of teachers. Like our study, the authors use the MET data and perform a number of methodological approaches to identify whether matching of…”
BackgroundMethodInfluentialGoldhaber, D., Lavery, L., & Theobald, R. (2015) · Educational Researcher · doi:10.3102/0013189x15592622 · Cited by 351
(2013) · Cited by 345
Incentives, Selection, and Teacher Performance: Evidence from IMPACT
Dee, T. S., & Wyckoff, J. (2015) · Journal of Policy Analysis and Management · doi:10.1002/pam.21818 · Cited by 331
Research works printed but not linked: 10
- American Educational Research Association, 2015, Educational Researcher, 44
- Baker E. L., 2010, Problems with the use of student test scores to evaluate teachers
- Cornell S., 2006, Ethnicity and race: Making identities in a changing world
- Ho A. D., 2013, The reliability of classroom observations by school personnel
- Jiang J. Y., 2016, Teacher evaluation in Chicago: Differences in observation and value-added scores by teacher, student, and school characteristics
- Kane T. J., 2012, Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains
- Kane T. J., 2011, Journal of Human Resources, 46, 587
- Sanders W., 1996, Cumulative and residual effects of teachers on future student academic achievement: Research progress report
- White M., 2014, User guide to measures of effective teaching longitudinal database (MET LDB)
- Whitehurst G. J. R., 2014, Evaluating teachers with classroom observations
Cited by
What difference does school leadership make to student outcomes?
Robinson, V., & Gray, E. (2019) · Journal of the Royal Society of New Zealand · doi:10.1080/03036758.2019.1582075 · Cited by 109
“through classroom observations of teaching or value-added measures of student growth, is not yet sufficiently valid to justify high stakes decisions about teacher performance (Campbell and Ronfeldt 2018).”
BackgroundFauth, B., Wagner, W., Bertram, C., Göllner, R., Roloff, J., Lüdtke, O., Polikoff, M. S., Klusmann, U., & Trautwein, U. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000416 · Cited by 120
Understanding, Choosing, and Unifying Multilevel and Fixed Effect Approaches
Hazlett, C. J., & Wainstein, L. (2020) · Political Analysis · doi:10.1017/pan.2020.41 · Cited by 75
Ratings in black and white: a quantcrit examination of race and gender in teacher evaluation reform
Campbell, S. L. (2020) · Race Ethnicity and Education · doi:10.1080/13613324.2020.1842345 · Cited by 57
Self-citationTeacher Skill Development: Evidence from Performance Ratings by Principals
Kraft, M. A., Papay, J. P., & Chi, O. L. (2019) · Journal of Policy Analysis and Management · doi:10.1002/pam.22193 · Cited by 62
“Recent research suggests that teachers’ classroom composition can influence the observation scores they receive (Whitehurst, Chingos, & Lindquist, 2014; Steinberg & Garrett, 14 2016; Campbell & Ronfeldt, 2018).”
Background
Metadata
- Authors
- Campbell, S. L., & Ronfeldt, M.
- Affiliations
- University of California, Irvine (US)
- University of Michigan (US)
- Corresponding author
- Campbell, Shanyce L.
- Venue
- American Educational Research Journal
- Published
- 2018-06-07
- Volume
- 55
- Issue
- 6
- Pages
- 1233-1267
- ISSN
- 0002-8312 (ISSN-L)
- 1935-1011
- OpenAlex
- W2556979109
- Cited by
- 111
- References
- 24
- Citations by year
- 2026: 4 · 2025: 9 · 2024: 14 · 2023: 22 · 2022: 18 · 2021: 16 · 2020: 18 · 2019: 10
- FWCI
- 35.67
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- American Educational Research Journal (publishedVersion)
- 2016 Fall Conference: The Role of Research in Making Government More Effective
- Topics
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Education › Teacher Education and Assessments
- Social Sciences › Education › Educational Assessment and Improvement
- Keywords
- teacher evaluation, Measures of Effective Teaching, teacher quality, student concentration, underachieving students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Observational Evaluation of Teachers: Measuring More Than We Bargained for? https://app.cochat.ai/papers/10.3102/0002831218776216
- The paper
- Campbell, S. L., & Ronfeldt, M. (2018). Observational Evaluation of Teachers: Measuring More Than We Bargained for? American Educational Research Journal. https://doi.org/10.3102/0002831218776216
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.