Rater Performance Standards for Classroom Observation Instruments
The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Raters must score accurately and consistently for classroom observation scores to be valid. This requires (a) a standard defining when scoring is accurate and consistent enough and (b) measuring and remediating rater performance against that standard. Current practice has focused on this second problem to the exclusion of the first. My goal here is to start a discussion about identifying a clear, explicit standard that ensures observation scores reflect a consistent view of teaching quality, rather than raters’ idiosyncratic perspectives. In doing so, I connect current certification test cut-scores, the current practice most analogous to a standard, to explicit rater standards, highlighting both the inadequacy of cut-scores and the low standards implicit to current practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The standards for educational and psychological testing.
Eignor, D. R. (2013) · APA handbook of testing and assessment in psychology, Vol. 1: Test theory and testing and assessment in industrial and organizational psychology. · doi:10.1037/14047-013 · Cited by 6,988
Enhancing Professional Practice: A Framework for Teaching
Danielson, C. (1996) · Cited by 1,576
Humphreys, L. G., & Lindquist, E. F. (1952) · The American Journal of Psychology · doi:10.2307/1418848 · Cited by 1,502
Classroom emotional climate, student engagement, and academic achievement.
Reyes, M. R., Brackett, M. A., Rivers, S. E., White, M., & Salovey, P. (2012) · Journal of Educational Psychology · doi:10.1037/a0027268 · Cited by 1,229
Self-citationValidating the Performance Standards Associated with Passing Scores
Kane, M. (1994) · Review of Educational Research · doi:10.2307/1170678 · Cited by 288
Grossman, P., Loeb, S., Cohen, J., & Wyckoff, J. (2013) · American Journal of Education · doi:10.1086/669901 · Cited by 271
An Argument Approach to Observation Protocol Validity
Bell, C. A., Gitomer, D. H., McCaffrey, D. F., Hamre, B. K., Pianta, R. C., & Qi, Y. (2012) · Educational Assessment · doi:10.1080/10627197.2012.715014 · Cited by 236
(2014) · Cited by 207
Building a More Complete Understanding of Teacher Evaluation Using Classroom Observations
Cohen, J., & Goldhaber, D. (2016) · Educational Researcher · doi:10.3102/0013189x16659442 · Cited by 201
(2012) · Routledge eBooks · doi:10.4324/9780203848203 · Cited by 157
Research works printed but not linked: 3
- Pianta R. C., 2010, Classroom Assessment Scoring System (CLASS): Upper elementary manual
- Rowan B., 2013, Promoting high quality teacher evaluations in Michigan: Lessons from a pilot of educator effectiveness tools
- White M. C., 2017, The generalizability of scores from observation instruments
Cited by
Ensuring content validity of psychological and educational tests – the role of experts
Beck, K. (2020) · Frontline Learning Research · doi:10.14786/flr.v8i6.517 · Cited by 85
“Nevertheless, White’s (2018) sophisticated discussion on”
Ratings in black and white: a quantcrit examination of race and gender in teacher evaluation reform
Campbell, S. L. (2020) · Race Ethnicity and Education · doi:10.1080/13613324.2020.1842345 · Cited by 57
“While these recommendations attempt to ensure that administrators are trained and certified, there is still the likelihood that principal error will occur (White 2018).”
BackgroundClassroom observation systems in context: A case for the validation of observation systems
Liu, S., Bell, C. A., Jones, N. D., & McCaffrey, D. F. (2019) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-018-09291-3 · Cited by 46
Kelly, S., Bringe, R., Aucejo, E. M., & Fruehwirth, J. C. (2020) · Education Policy Analysis Archives · doi:10.14507/epaa.28.5012 · Cited by 55
Klette, K. (2023) · The Journal of Curriculum Studies · doi:10.1080/00220272.2023.2172360 · Cited by 25
“Several scholars (Gitomer, 2009; M. C. White, 2018; M. White & Ronfeldt, 2020) have noted how rater error is a major impediment for using observation manuals in a reliable and valid way.”
Background
Metadata
- Authors
- White, M. C.
- Affiliations
- University of Michigan (US)
- Corresponding author
- White, Mark C.
- ORCID
- White, Mark C.
- Venue
- Educational Researcher
- Published
- 2018-07-09
- Volume
- 47
- Issue
- 8
- Pages
- 492-501
- ISSN
- 0013-189X (ISSN-L)
- 1935-102X
- OpenAlex
- W2835004802
- Semantic Scholar
- 55354c8a072abe3047339e956b5559d14163ea81
- Cited by
- 32
- References
- 20
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 5 · 2023: 3 · 2022: 10 · 2021: 1 · 2020: 8 · 2019: 3
- FWCI
- 9.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Educational Researcher (publishedVersion)
- Topics
- Social Sciences › Education › Teacher Education and Assessments
- Social Sciences › Education › Evaluation of Teaching Practices
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- classroom observation, rater reliability, rater validity, scoring consistency, cut-off values, teacher certification, teaching quality assessment, observation instruments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Rater Performance Standards for Classroom Observation Instruments. https://app.cochat.ai/papers/10.3102/0013189x18785623
- The paper
- White, M. C. (2018). Rater Performance Standards for Classroom Observation Instruments. Educational Researcher. https://doi.org/10.3102/0013189x18785623
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.