What Are Value-Added Models Estimating and What Does This Imply for Statistical Practice?

Abstract

The question of how to estimate school and teacher contributions to student learning is fundamental to educational policy and practice, and the three thoughtful articles in this issue represent a major advance. The current level of public confusion about these issues is so severe and the consequences for schooling so great that it is a big relief to see this journal highlight the key issues. A common theme in these articles is that we should compare schools or teachers by comparing their value added to student learning rather than by comparing unadjusted mean levels of achievement or, as is currently common practice, the percent of students in a school or class who are classified as proficient. As Ballou, Sanders, and Wright (BSW) note, it makes no sense to hold schools accountable for mean achievement levels when students enter those schools with large mean differences in achievement. Moreover, given the remarkable mobility of students across schools, particularly in large urban districts, changes in mean achievement at the school level may bear little relation to instructional effectiveness. In contrast, the value-added philosophy is to hold schools and teachers accountable for the learning gains of students they serve. This seems simple enough, yet the technical questions raised in these articles are many: whether and how to adjust for covariates, whether teachers (or schools) should be treated as fixed or random, how to represent cumulative effects of teachers or schools, how to model covariation in student responses and teacher effects, whether and how to incorporate multiple cohorts, and how to formulate models that appropriately handle missing data. A prior question is: What are we trying to estimate with these models? School and teacher effects are causal effects, yet the treatments students experience and the potential outcomes under alternative treatments (Rubin, 1978; Rosenbaum & Rubin, 1983; Holland, 1986) are not clearly defined in these discussions. As a