Using Classroom Observations in the Evaluation of Special Education Teachers
The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Abstract
We examine whether one of the most popular observation systems in teacher evaluation—the Framework for Teaching (FFT)—captures the range of instructional skills teachers need to be effective. We focus on the case of special educators, who are likely to use instructional approaches that, although supported by research, are de-emphasized in common observation systems. Drawing on 206 lessons from 51 teachers, we compare FFT scores to an observation system from special education. We find that FFT’s psychometric properties are consistent with previous studies, but the system is limited in assessing the quality of instructional practices used in special education. We discuss implications of these findings for two practical uses of observations—supporting teacher development and informing human capital decisions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Cognitive Load During Problem Solving: Effects on Learning
Sweller, J. (1988) · Cognitive Science · doi:10.1016/0364-0213(88)90023-7 · Cited by 11,048 · cited 1×
“What explains these findings? In research on cognition, it is well established that the cognitive demands of learning are higher when students have limited prior knowledge; when novice learners encounter new problems, they have to dedicate more working memory to coding new information, while experts have the schema ne…”
BackgroundKirschner, P. A., Sweller, J., & Clark, R. E. (2006) · Educational Psychologist · doi:10.1207/s15326985ep4102_1 · Cited by 7,776 · cited 1×
“What explains these findings? In research on cognition, it is well established that the cognitive demands of learning are higher when students have limited prior knowledge; when novice learners encounter new problems, they have to dedicate more working memory to coding new information, while experts have the schema ne…”
BackgroundSearle, S. R., Casella, G., & McCulloch, C. E. (1992) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316856 · Cited by 1,926
Enhancing Professional Practice: A Framework for Teaching
Danielson, C. (1996) · Cited by 1,576
Humphreys, L. G., & Lindquist, E. F. (1952) · The American Journal of Psychology · doi:10.2307/1418848 · Cited by 1,502
Generalizability theory: a primer
(1992) · Choice Reviews Online · doi:10.5860/choice.29-4192 · Cited by 1,298
Brennan, R. L. (2001) · doi:10.1007/978-1-4757-3456-0 · Cited by 1,121
Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction
Kalyuga, S. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9054-3 · Cited by 1,056 · cited 1×
“This idea is referred to in the literature as the expertise reversal effect (Kalyuga, 2007), or alternatively aptitude treatment interaction (Snow, 1989), or more recently, childinstruction interaction (e.”
Background(2009) · Issue Lab (Candid) · Cited by 552
Jacob, B. A., & Lefgren, L. (2008) · Journal of Labor Economics · doi:10.1086/522974 · Cited by 494
Learning and Individual Differences
Chen, J., & Usher, E. L. (2013) · Cited by 469
The Underlying Message in LD Intervention Research: Findings from Research Syntheses
Vaughn, S., Gersten, R., & Chard, D. J. (2000) · Exceptional Children · doi:10.1177/001440290006700107 · Cited by 394
The Effectiveness of Direct Instruction Curricula: A Meta-Analysis of a Half Century of Research
Stockard, J., Wood, T. W., Coughlin, C., & Rasplica Khoury, C. (2018) · Review of Educational Research · doi:10.3102/0034654317751919 · Cited by 371 · cited 2×
“Studies suggest that all students benefit from explicit, systematic instruction (Clements et al., 2013; Cohen, 2018; Stockard et al., 2018; Tyler, 2018), with such instruction being particularly beneficial among lower performing students (e.”
BackgroundHill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012) · Educational Researcher · doi:10.3102/0013189x12437203 · Cited by 331
Effective Reading Comprehension Instruction: Examining Child x Instruction Interactions.
Connor, C. M., Morrison, F. J., & Petrella, J. N. (2004) · Journal of Educational Psychology · doi:10.1037/0022-0663.96.4.682 · Cited by 316
One lesson is all you need? Stability of instructional quality across lessons
Praetorius, A.-K., Pauli, C., Reusser, K., Rakoczy, K., & Klieme, E. (2014) · Learning and Instruction · doi:10.1016/j.learninstruc.2013.12.002 · Cited by 313
Extensive Reading Interventions for Students With Reading Difficulties After Grade 3
Wanzek, J., Vaughn, S., Scammacca, N. K., Metz, K., Murray, C. S., Roberts, G., & Danielson, L. (2013) · Review of Educational Research · doi:10.3102/0034654313477212 · Cited by 276
Hill, H., & Grossman, P. (2013) · Harvard Educational Review · doi:10.17763/haer.83.2.d11511403715u376 · Cited by 255
An Argument Approach to Observation Protocol Validity
Bell, C. A., Gitomer, D. H., McCaffrey, D. F., Hamre, B. K., Pianta, R. C., & Qi, Y. (2012) · Educational Assessment · doi:10.1080/10627197.2012.715014 · Cited by 236
Self-citationAlgorithm-Guided Individualized Reading Instruction
Connor, C. M., Morrison, F. J., Fishman, B. J., Schatschneider, C., & Underwood, P. (2007) · Science · doi:10.1126/science.1134513 · Cited by 225
Kraft, M. A., & Gilmour, A. F. (2017) · Educational Researcher · doi:10.3102/0013189x17718797 · Cited by 211
(2014) · Cited by 207
Efficacy of Collaborative Strategic Reading With Middle School Students
Vaughn, S., Klingner, J. K., Swanson, E. A., Boardman, A. G., Roberts, G., Mohammed, S. S., & Stillman-Spisak, S. J. (2011) · American Educational Research Journal · doi:10.3102/0002831211410305 · Cited by 202
Fuchs, L. S., Powell, S. R., Seethaler, P. M., Cirino, P. T., Fletcher, J. M., Fuchs, D., Hamlett, C. L., & Zumeta, R. O. (2009) · Journal of Educational Psychology · doi:10.1037/a0014701 · Cited by 196
Steinberg, M. P., & Donaldson, M. L. (2016) · Education Finance and Policy · doi:10.1162/edfp_a_00186 · Cited by 183
Handbook of Research on Special Education Teacher Preparation
(2014) · Routledge eBooks · doi:10.4324/9780203817032 · Cited by 178
Xue, Y., & Meisels, S. J. (2004) · American Educational Research Journal · doi:10.3102/00028312041001191 · Cited by 165
Qualities of classroom observation systems
Bell, C. A., Dobbelaer, M. J., Klette, K., & Visscher, A. (2019) · School Effectiveness and School Improvement · doi:10.1080/09243453.2018.1539014 · Cited by 160
Self-citationThe Role of Domain Expertise in Beginning Special Education Teacher Quality
Brownell, M. T., Bishop, A. G., Gersten, R. M., Klingner, J. K., Penfield, R. D., Dimino, J., Haager, D., Menon, S., & Sindelar, P. T. (2009) · Exceptional Children · doi:10.1177/001440290907500401 · Cited by 151
Self-citationSkills, productivity and the evaluation of teacher performance
Harris, D. N., & Sass, T. R. (2014) · Economics of Education Review · doi:10.1016/j.econedurev.2014.03.002 · Cited by 142
Al Otaiba, S., Connor, C. M., Folsom, J. S., Greulich, L., Meadows, J., & Li, Z. (2011) · The Elementary School Journal · doi:10.1086/659031 · Cited by 137
Investigating Instructional Leadership for Special Education
Bays, D. A., & Crockett, J. B. (2007) · Exceptionality · doi:10.1080/09362830701503495 · Cited by 116
Hamre, B. K., Pianta, R. C., Mashburn, A. J., & Downer, J. T. (2012) · Early Education & Development · doi:10.1080/10409289.2011.607360 · Cited by 107
Instruction, Student Engagement, and Reading Skill Growth in Reading First Classrooms
Connor, C. M., Jakobsons Lara, J., Crowe, E. C., & Meadows, J. G. (2008) · The Elementary School Journal · doi:10.1086/592305 · Cited by 106
Trends in Classroom Observation Scores
Casabianca, J. M., Lockwood, J. R., & McCaffrey, D. F. (2015) · Educational and Psychological Measurement · doi:10.1177/0013164414539163 · Cited by 105 · cited 1×
“As many scholars have suggested (Casabianca et al., 2015; Cash & Pianta, 2014; Mashburn, Downer, et al., 2014; Meyer et al., 2011), care should be taken to map a sample of lessons onto the population of a teachers’ lessons over the course of the year, being mindful, for example, to not oversample lessons from any one…”
BackgroundExamining the Use of Classroom Observations in the Evaluation of Special Education Teachers
Jones, N. D., & Brownell, M. T. (2013) · Assessment for Effective Intervention · doi:10.1177/1534508413514103 · Cited by 105
Self-citationDonaldson, M. L., & Woulfin, S. (2018) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373718784205 · Cited by 105
How Framework for Teaching and Tripod 7Cs Evidence Distinguish Key Components of Effective Teaching
Ferguson, R. F., & Danielson, C. (2015) · Wiley eBooks · doi:10.1002/9781119210856.ch4 · Cited by 104
Morgan, P. L., Farkas, G., & Maczuga, S. (2015) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373714536608 · Cited by 96
Once, Sometimes, or Always in Special Education: Mathematics Growth and Achievement Gaps
Schulte, A. C., & Stevens, J. J. (2015) · Exceptional Children · doi:10.1177/0014402914563695 · Cited by 94
Smolkowski, K., & Gunn, B. (2011) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2011.09.004 · Cited by 90
Vadasy, P. F., Sanders, E. A., & Peyton, J. A. (2005) · Journal of Learning Disabilities · doi:10.1177/00222194050380041401 · Cited by 89
Cohen, J. (2017) · Teaching and Teacher Education · doi:10.1016/j.tate.2017.10.021 · Cited by 78
Doabler, C. T., Baker, S. K., Kosty, D. B., Smolkowski, K., Clarke, B., Miller, S. J., & Fien, H. (2015) · The Elementary School Journal · doi:10.1086/679969 · Cited by 77 · cited 4×
“Once all recordings were complete, individual lessons were scored using two observation systems: the FFT and the QCI (Doabler et al., 2015).”
BackgroundMethodInfluentialSchulte, A. C., Stevens, J. J., Elliott, S. N., Tindal, G. A., & Nese, J. F. T. (2016) · Journal of Educational Psychology · doi:10.1037/edu0000107 · Cited by 77
Sonnenschein, S., Stapleton, L. M., & Benson, A. (2010) · American Educational Research Journal · doi:10.3102/0002831209349215 · Cited by 73
Explaining the Decline in Special Education Teacher Employment From 2005 to 2012
Dewey, J., Sindelar, P. T., Bettini, E. A., Boe, E. E., Rosenberg, M. S., & Leko, C. D. (2017) · Exceptional Children · doi:10.1177/0014402916684620 · Cited by 70
Using assessment to individualize early mathematics instruction
Connor, C. M., Mazzocco, M. M. M., Kurz, T., Crowe, E. C., Tighe, E. L., Wood, T. S., & Morrison, F. J. (2018) · Journal of School Psychology · doi:10.1016/j.jsp.2017.04.005 · Cited by 68
Mashburn, A. J., Downer, J. T., Rivers, S. E., Brackett, M. A., & Martínez, A. (2013) · Prevention Science · doi:10.1007/s11121-012-0357-3 · Cited by 61
Individualizing Student Instruction in Reading: Implications for Policy and Practice
Connor, C. M., & Morrison, F. J. (2016) · Policy Insights from the Behavioral and Brain Sciences · doi:10.1177/2372732215624931 · Cited by 56
Mantzicopoulos, P. Y., French, B. F., Patrick, H., Watson, J. S., & Ahn, I. (2017) · Educational Assessment · doi:10.1080/10627197.2017.1408407 · Cited by 56
Improving Observational Score Quality
Bell, C. A., Qi, Y., Croft, A. J., Leusner, D., McCaffrey, D. F., Gitomer, D. H., & Pianta, R. C. (2015) · Wiley eBooks · doi:10.1002/9781119210856.ch3 · Cited by 55
Self-citationEmpowering Teachers and Students with Instructional Choices in Inclusive Settings
MERCER, C. D., Lane, H. B., Jordan, L., Allsopp, D. H., & Eisele, M. R. (1996) · Remedial and Special Education · doi:10.1177/074193259601700405 · Cited by 49
Meyer, J. P., Cash, A. H., & Mashburn, A. J. (2011) · Educational Assessment · doi:10.1080/10627197.2011.638884 · Cited by 49
Le, V., Stecher, B. M., Lockwood, J. R., III, et al. (2006) · PsycEXTRA Dataset · doi:10.1037/e654632010-001 · Cited by 48
Classroom observation systems in context: A case for the validation of observation systems
Liu, S., Bell, C. A., Jones, N. D., & McCaffrey, D. F. (2019) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-018-09291-3 · Cited by 46
Self-citationStelitano, L., Russell, J. L., & Bray, L. E. (2019) · American Educational Research Journal · doi:10.3102/0002831219859307 · Cited by 46
Baker, S. K., Gersten, R., Haager, D., & Dingle, M. (2006) · The Elementary School Journal · doi:10.1086/510655 · Cited by 43
Measuring Rater Reliability on a Special Education Observation Tool
Semmelroth, C., & Johnson, E. S. (2013) · Assessment for Effective Intervention · doi:10.1177/1534508413511488 · Cited by 39
Uncovering Multivariate Structure in Classroom Observations in the Presence of Rater Errors
McCaffrey, D. F., Yuan, K., Savitsky, T. D., Lockwood, J. R., & Edelen, M. O. (2015) · Educational Measurement: Issues and Practice · doi:10.1111/emip.12061 · Cited by 38
Perspectives on the Validity of Classroom Assessments
Kane, M. T., & Wools, S. (2019) · Routledge eBooks · doi:10.4324/9780429507533-2 · Cited by 33
Morris-Mathews, H., Stark, K. R., Jones, N. D., Brownell, M. T., & Bell, C. A. (2021) · Journal of Learning Disabilities · doi:10.1177/0022219420941804 · Cited by 30 · cited 4×
“These findings support evidence from a content study of the FFT’s Instruction domain that showed that FFT scoring criteria and descriptions do not fully capture effective special education instruction (Morris-Mathews et al., 2020).”
BackgroundResultInfluentialSelf-citationMemory and Information Processes
Mayer, R. E. (2003) · Handbook of Psychology · doi:10.1002/0471264385.wei0703 · Cited by 19
Strategies for Assessing Classroom Teaching: Examining Administrator Thinking as Validity Evidence
Bell, C. A., Jones, N. D., Qi, Y., & Lewis, J. M. (2018) · Educational Assessment · doi:10.1080/10627197.2018.1513788 · Cited by 18
Self-citationJohnson, E. S., Zheng, Y. Z., Crawford, A., & Moylan, L. A. (2020) · Journal of Learning Disabilities · doi:10.1177/0022219420908906 · Cited by 18
Taylor, E. S. (2017) · Journal of Labor Economics · doi:10.1086/696144 · Cited by 15
Chudowsky, N., & Chudowsky, V. (2009) · Center on Education Policy · Cited by 14
Teaching Students With Special Needs in the New Millennium
Klingner, J. K., Brownell, M. T., Mason, L. H., et al. (2016) · doi:10.3102/978-0-935302-48-6_10 · Cited by 14
Self-citationThe Role of Scheduling in Observing Teacher–Child Interactions
Cash, A. H., & Pianta, R. C. (2014) · School Psychology Review · doi:10.1080/02796015.2014.12087414 · Cited by 12
Bettini, E. A., Mason-Williams, L., & Barber, B. R. (2019) · The Journal of Special Education · doi:10.1177/0022466919831302 · Cited by 11
Gilmour, A. F., & Jones, N. D. (2020) · Educational Researcher · doi:10.3102/0013189x20935039 · Cited by 11
Self-citationOliveri, M. E., McCaffrey, D. F., Ezzo, C., & Holtzman, S. L. (2017) · Applied Measurement in Education · doi:10.1080/08957347.2017.1353989 · Cited by 9
Self-citationDoes Special Educator Effectiveness Vary Depending on the Observation Instrument Used?
Johnson, E. S., Crawford, A., Zheng, Y. Z., & Moylan, L. A. (2020) · Educational Measurement Issues and Practice · doi:10.1111/emip.12394 · Cited by 7
The Role of Scheduling in Observing Teacher–Child Interactions
Cash, A. H., & Pianta, R. C. (2014) · School Psychology Review · doi:10.17105/spr-13-0033.1 · Cited by 6
The relationship of cross-modal memory tasks with other neuropsychological measures
Snow, J. H. (1989) · Learning and Individual Differences · doi:10.1016/1041-6080(89)90017-4
Teacher Demand, Supply, and Shortage in Special Education
Boe, E. E. (n.d.) · Handbook of Research on Special Education Teacher Preparation · doi:10.4324/9780203817032.ch5
Research works printed but not linked: 11
- Bell C. A., 2016, Understanding consequential assessment systems of teaching: Year 1 final report to Los Angeles Unified School District
- Briggs D. C., 2014, Comparing student growth and teacher observation to principal judgments in the evaluation of teacher effectiveness
- Clements D. H., 2013, Instructional practices and student achievement: Correlations from a study of math curricula
- Croft M., 2018, The shrinking use of growth: Teacher evaluation legislation since ESSA
- Cronbach L. J., 1988, Test validity, 3
- Cronbach L. J., 1972, The dependability of behavioral measurement: Theory of generalizability for scores and profiles
- Danielson C., 2013, The framework for teaching evaluation instrument, 2013 edition
- Ho A. D., 2013, The reliability of classroom observations by school personnel
- Kane T. J., 2012, Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains
- Martin A. J., 2016, Using load reduction instruction (LRI) to boost motivation and engagement
- Steinbrecher T. D., 2015, Journal of Special Education Leadership, 28, 89
Cited by
Observation Studies in Special Education: A Synthesis of Validity Evidence for Observation Systems
Rodgers, W., Mathews, H. M., Romig, J. E., & Bettini, E. A. (2021) · Review of Educational Research · doi:10.3102/00346543211042419 · Cited by 17
“33 its use with K–12 special educators (Jones et al., 2021).”
BackgroundObservational Record and Self-Report of Teacher-Student Performance in High School Lessons
Bazán-Ramírez, A., Velarde, N., Pérez, M. E. R., Guerrero-Barrios, J., & Anaya-González, R. B. (2022) · International Journal of Educational Methodology · doi:10.12973/ijem.8.3.479 · Cited by 8
“One relevant topic to discuss is what good teaching looks like since Jones et al. (2022) have concluded that the use of universal teaching performance criteria may result in unfair teacher evaluations.”
BackgroundGitomer, D. H., & Marshall, B. L. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1221569 · Cited by 7
Are Effective Teachers for Students With Disabilities Effective Teachers for All?
Wood, W. J., Lai, I., Imberman, S. A., Jones, N., Strunk, K., Liang, C., Ma, W., Tong, H., & Shi, Z. (2022) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737231214555 · Cited by 8
“One possibility, supported by existing literature, is that some teachers are more effective than others at tailoring their instruction—within the classroom— to the unique needs of different learners (see Jones et al., 2022, or Connor et al., 2018, for example).”
InfluentialSelf-citationBruhn, J., Jones, N. D., Kanno, Y., & Winters, M. A. (2022) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737221136101 · Cited by 3
“Some of the strategies emphasized by the course, such as modeling and explicit instruction, have also been found to benefit struggling readers (Connor et al., 2011), students with disabilities (Jones et al., 2022), and even the larger general student population (Cohen, 2018).”
BackgroundSelf-citation
Metadata
- Authors
- Jones, N. D., Bell, C. A., Brownell, M., Qi, Y., Peyton, D., Pua, D., Fowler, M., & Holtzman, S.
- Affiliations
- Boston University (US)
- University of Wisconsin–Madison (US)
- University of Florida (US)
- Educational Testing Service (US)
- Appalachian State University (US)
- Corresponding author
- Jones, Nathan D.
- Venue
- Educational Evaluation and Policy Analysis
- Published
- 2022-01-31
- Volume
- 44
- Issue
- 3
- Pages
- 429-457
- ISSN
- 0162-3737 (ISSN-L)
- 1935-1062
- OpenAlex
- W4210305525
- Cited by
- 21
- References
- 71
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 4 · 2023: 5 · 2022: 2
- FWCI
- 8.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Educational Evaluation and Policy Analysis (publishedVersion)
- Funders
- Institute of Education Sciences (R324A150231)
- Topics
- Social Sciences › Education › Teacher Education and Assessments
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Education › Evaluation of Teaching Practices
- Keywords
- teacher evaluation, classroom observation, special education teachers, instructional practices, psychometric properties, teacher professional development, special education instruction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.