Once is not enough: Establishing reliability criteria for feedback and evaluation decisions based on classroom observations
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012) · Phi Delta Kappan · doi:10.1177/003172171209300603 · Cited by 402 · see citing works
Teacher Evaluation Policy and Conflicting Theories of Motivation
Firestone, W. A. (2014) · Educational Researcher · doi:10.3102/0013189x14521864 · Cited by 159 · cited 1× · see citing works
“Of these two purposes of teacher evaluation, the decisions regarding teacher selection currently receive the most attention (e.g., Firestone, 2014; Winter & Cowen, 2014).”
Background
Flagged first, then by citations
Research works printed but not linked: 20
- Brennan, R. L. (2001). Generalizability theory: statistics for social science and public policy. New York: Springer-Verlag.
- Briggs, D. C., & Wislon, M. (2007). Generalizability in item response theory. Journal of Educational Measurement, 44, 131-155.
- Centra, J. A. (1975). Colleagues as raters of classroom instruction. The Journal of Higher Education, 46(3), 327-337. http://dx.doi.org/10.2307/1980806. · DOI
- Choi, J. (2012). Advances in combining generalizability theory and item response theory. Doctoral dissertation. Berkeley: University of California.
- De Boeck, P., Bakker, M., Zwitser, R., Nivard, M., Abe, H., Tuerlinckx, F., et al. (2011). The estimation of item response models with the lmer function from the lme4 package in R. Journal of Statistical Software, 39, 1-25.
- French-Lazovik, G. (1981). Documentary evidence in the evaluation of teaching. In J. Millman (Ed.), Handbook of teacher evaluationBeverly Hills, CA: Sage Publications.
- Hill, H., Charalambous, C. Y., & Kraft, M. A. (2012). When interrater-reliability is not enough: Teacher observation systems and a case for the generalizability theory. Educational Researcher, 41, 56-64. http://dx.doi.org/10.3102/ 0013189x12437203. · DOI · Paper page
- Kane, M. T. (2006). Validation, In R. L. Brennan (Ed.), Educational measurement (pp. 17-64).4th ed. Washington, DC: The National Council on Measurement in Education & the American Council on Education.
- Muijs, D. (2006). Measuring teacher effectiveness: Some methodological reflections. Educational Research and Evaluation: An International Journal on Theory and Practice, 12, 53-74.
- Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill. OCW [Dutch Ministry of Education, Culture, Science] (2013). Peer review in de praktijk [Peer review in practice]. Rotterdam: VOION.
- Peterson, K. D., & Chenoweth, T. (1992). School teachers' control and involvement in their own evaluation. Journal of Personnel Evaluation in Education, 6(2), 177-189.
- Peterson, K. D. (2000). Teacher evaluation: a comprehensive guide to new directions and practice. Thousand Oaks, CA: Corwin press.
- Popham (1988). The dysfunctional marriage of formative and summative teacher evaluation. Journal of Personnel Evaluation in Education, 1, 269-273.
- Raju, N. S., Price, L. R., Oshima, T. C., & Nering, M. L. (2006). Standardized conditional SEM: a case for conditional reliability. Applied Psychological Measurement, 30, 1- 12. http://dx.doi.org/10.1177/0146621606291569. · DOI
- Scriven, M. (1981). Summative teacher evaluation. In J. Millman (Ed.), Handbook of teacher evaluationBeverly Hills, CA: Sage Publications.
- Shavelson, R., & Dempsey-Atwood, N. (1976). Generalizability of teaching behavior. Review of Educational Research, 46, 553-611.
- Strong, M. (2011). The highly qualified teacher: what is teacher quality and how do we measure it? NY, New York: Teachers College Press.
- van de Grift, W. J. C. M. (2013). Measuring teaching quality in several European countries. School effectiveness and school improvement, 25(3) . http://dx.doi.org/ 10.1080/09243453.2013.794845. · DOI
- van de Grift, W. J. C. M., Helms-Lorenz, M., & Maulana, R. (2014). Teaching skills of student teachers: Calibration of an evaluation instrument and its value in predicting student academic engagement. Studies in Educational Evaluation, 43, 150-159. http://dx.doi.org/10.1016/j.stueduc.2014.09.003. · DOI
- Winters, M. A., & Cowen, J. M. (2014). Who would stay, who would be dismissed? An empirical consideration of value-added teacher retention policies. Educational Researcher, 42, 330-337. http://dx.doi.org/10.3102/0013189x13496145. · DOI
Cited by
Sorted by citations
Classroom observation systems in context: A case for the validation of observation systems
Liu, S., Bell, C. A., Jones, N. D., & McCaffrey, D. F. (2019) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-018-09291-3 · Cited by 46 · see citing works
Exploring Differences in Measurement and Reporting of Classroom Observation Inter-Rater Reliability
Wilhelm, A. G., Rouse, A. G., & Jones, F. (2018) · University of Massachusetts Amherst · doi:10.7275/at67-md25 · Cited by 20 · see citing works
Weston, T. J., Hayward, C. N., & Laursen, S. L. (2021) · American Journal of Evaluation · doi:10.1177/1098214020931941 · Cited by 20 · see citing works
- Authors
- van der Lans, R. M., van de Grift, W. J. C. M., van Veen, K., & Fokkens-Bruinsma, M.
- Affiliations
- University of Groningen (NL)
- Venue
- Studies in Educational Evaluation
- Publisher
- Elsevier BV
- Published
- 2016-09
- Volume
- 50
- Pages
- 88-95
- ISSN
- 0191-491X (ISSN-L)
- OpenAlex
- W2485286459
- Cited by
- 55
- References
- 2
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 4 · 2023: 6 · 2022: 2 · 2021: 12 · 2020: 6 · 2019: 6 · 2018: 5 · 2017: 6 · 2016: 1 · 2015: 1
- FWCI
- 24.03
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Studies In Educational Evaluation (publishedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
- Topics
- Social Sciences › Education › Teacher Education and Assessments
- Social Sciences › Education › Evaluation of Teaching Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Educational Assessment and Improvement
- Keywords
- teacher evaluation, classroom observation, reliability statistics, feedback, number of observations, peer observation, teacher professional development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
