On the evaluation of systemic science education reform: Searching for instructional sensitivity

Abstract

We propose a multilevel‐multifaceted approach to evaluating the impact of education reform on student achievement that would be sensitive to context and small treatment effects. The approach uses different assessments based on their proximity to the enacted curriculum. Immediate assessments are artifacts (students' products) from the enactment of the curriculum; close assessments parallel the content and activities of the unit/curriculum; proximal assessments tap knowledge and skills relevant to the curriculum, but topics can be different; and distal assessments reflect state/national standards in a particular knowledge domain. To provide evidence about the sensitivity of the multilevel approach in ascertaining outcomes of hands‐on science programs we administered close, proximal, and distal performance assessments to evaluate the impact of instruction based on two Full Option Science System units—Variables, and Mixtures and Solutions—in a Bay Area school district. Results indicated that close assessments were more sensitive to the changes in students' pre‐ to post‐test performance than proximal assessments. © 2002 Wiley Periodicals, Inc. J Res Sci Teach 39: 369–393, 2002