CoChat

Classroom Composition and Measured Teacher Performance: What Do Teacher Observation Scores Really Measure?

Open access
Steinberg, Matthew P., Garrett, Rachel · 2016 · Educational Evaluation and Policy Analysis

The DOI matches its Crossref record; 34 of the 38 references it cites were found in a registry or an index; none of the 17 sources its claims cite is checked for support yet.

Abstract

As states and districts implement more rigorous teacher evaluation systems, measures of teacher performance are increasingly being used to support instruction and inform retention decisions. Classroom observations take a central role in these systems, accounting for the majority of teacher ratings upon which accountability decisions are based. Using data from the Measures of Effective Teaching study, we explore the extent to which classroom composition influences measured teacher performance based on classroom observation scores. The context in which teachers work—most notably, the incoming academic performance of their students—plays a critical role in determining teachers’ measured performance. Furthermore, the intentional sorting of teachers to students has a significant influence on measured performance. Implications for high-stakes teacher accountability policies are discussed.

Overview

CoChat Summary

TL;DR

Using Measures of Effective Teaching (MET) data, the paper finds that teachers’ observation-based performance scores rise when they are assigned higher-achieving students, even after accounting for teacher fixed effects. It also shows that nonrandom teacher-class matching can bias these observation scores, raising concerns about high-stakes evaluation systems that rely heavily on them.

Key findings

  • In the 2009–2010 school year, ELA teachers were “more than twice as likely” to be rated in the top performance quintile when assigned the highest-achieving students versus the lowest-achieving students; math teachers were “more than 6 times as likely.”
  • Roughly half of teachers assigned the highest-performing students were rated in the top two quintiles: 48% in ELA and 54% in math; among teachers assigned the lowest-performing students, the share was 37% in ELA and 18% in math.
  • In the full sample, a 1 standard deviation increase in incoming achievement was associated with a 0.11 FFT-point increase in ELA measured performance and a 0.06 FFT-point increase in math in the main estimate set; in the IV sample, the paper reports 0.14 FFT points for ELA and 0.08 FFT points for math.
  • For ELA teachers in partial-compliance classes, the OLS estimate was 0.21 FFT points and the IV estimate was 0.17 FFT points, both above the FE estimate of 0.14 FFT points, which the authors interpret as upward bias from nonrandom matching.
  • For specific practices, higher-achieving students were associated with better ratings on ELA instructional components 3a and 3c, while measures tied more to questioning and assessment (3b and 3d) were not sensitive to incoming achievement; math practices were largely unrelated to incoming achievement except for component 2b.

Methods

  • The study uses data from the MET project over two school years (2009–2010 and 2010–2011) across six districts, with an analytic teacher sample of 834 Grade 4–9 teachers (530 ELA, 480 math, with overlap).
  • Teacher performance is measured using classroom observation scores from the Danielson Framework for Teaching (FFT), aggregating eight observed components from video-recorded lessons.
  • The main analysis combines teacher fixed-effects models, OLS, and IV comparisons, and uses the Year 2 random assignment sample to assess causal effects and detect bias from nonrandom teacher-class matching.

Limitations

  • The paper states that classroom composition data were available only for teachers’ actual classes, not for the class rosters to which teachers were randomly assigned, and that student data for non-MET teachers after randomization were not observed.
  • The authors note that the FE approach can still be biased if there are time-varying shocks to teacher-class assignment, and the IV approach can be biased if there is partial/full noncompliance or selection into compliance classes.
  • Evident from the text: the experimental ELA result is described as “not precisely estimated,” and the authors say the math randomization sample did not satisfy the covariate-balance test, so the experimental estimate for math is not treated as a valid causal effect.

Model-written summary of a shortened version of the paper’s full text (it is longer than we read). Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3102/0162373715616249

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 2 Not valid · 10 Not checked · 104 Own results and interpretations

70 more claims

Create a free account to see all 116 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 22
  • Probably not a substantial claim. 23
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 34 references

Research works printed but not linked: 7

  1. Glazerman S., 2010, Evaluating teachers: The important role of value-added
  2. Goldhaber D. D., 2002, Education Next, 2, 50
  3. National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010, Common Core State Standards (Mathematics)
  4. Sartain L., Journal of Human Resources
  5. Steinberg M., Education Finance and Policy
  6. Watson J. G., 2009, The other 69 percent
  7. White M., 2012, A user guide to the “core study” data files available to MET early career grantees

Cited by

See all 150

Metadata

Authors
Steinberg, M. P., & Garrett, R.
Affiliations
  • University of Pennsylvania (US)
  • American Institutes for Research (US)
Venue
Educational Evaluation and Policy Analysis
Published
2016-06
Volume
38
Issue
2
Pages
293-317
ISSN
  • 0162-3737 (ISSN-L)
  • 1935-1062
OpenAlex
W2276065988
Cited by
150
References
38
Citations by year
2026: 4 · 2025: 9 · 2024: 8 · 2023: 16 · 2022: 20 · 2021: 17 · 2020: 26 · 2019: 21 · 2018: 13 · 2017: 12 · 2016: 3
FWCI
65.43
Type
journal article
Language
en
Open access
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
Educational Evaluation and Policy Analysis (publishedVersion)
Funders
National Academy of Education
Topics
  • Social Sciences › Education › School Choice and Performance
  • Social Sciences › Education › Teacher Education and Assessments
  • Social Sciences › Education › Educational Assessment and Improvement
Keywords
teacher evaluation, class composition, teacher performance ratings, classroom observation, academic performance, teacher sorting, teacher retention, teacher accountability, Measures of Effective Teaching, high-stakes testing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Classroom Composition and Measured Teacher Performance: What Do Teacher Observation Scores Really Measure? https://app.cochat.ai/papers/10.3102/0162373715616249
The paper
Steinberg, M. P., & Garrett, R. (2016). Classroom Composition and Measured Teacher Performance: What Do Teacher Observation Scores Really Measure? Educational Evaluation and Policy Analysis. https://doi.org/10.3102/0162373715616249

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.