Building and Supporting a Case for Test Use

Abstract

The fields of language testing and educational and psychological measurement have not, as yet, developed a set of principles and procedures for linking test scores and score-based inferences to test use and the consequences of test use. Although Messick (1989) discusses test use and consequences, his framework provides virtually no guidance on how to go about investigating these in the course of practical test development. Argument-based formulations of validity (e.g., Kane, 1992, 2000; Kane, Crooks, & Cohen, 1999; Mislevy, in press; Mislevy, Steinberg, & Almond, 2003) provide a logic and set of procedures for investigating and supporting claims about score-based inferences but do not address issues of test use and the consequences of test use. Recent formulations in language testing (e.g., Bachman & Palmer, 1996; Kunnan, 2003; Lynch, 2001) are essentially lists of more or less independent qualities and questions, with no clear mechanism for integrating these into a set of procedures for test developers and users to follow. What has been called "critical language testing" (e.g, Shohamy, 1999, 2001) has alerted us to the political uses and abuses of language tests and to the need for test developers and test users alike to be self-critical of the ways in which tests are used. However, this perspective treats consequences as essentially unrelated to the validity of inferences and provides little guidance about how to go about either anticipating and avoiding, or redressing, the problems with test use that it discusses. In this article I describe how an argument for test use might be structured so as to provide a clear linkage from test performance to interpretations and from interpretations to uses. An assessment use argument is an overall logical framework for linking assessment performance to use (decisions). This assessment use argument includes two parts: an assessment utilization argument, linking an interpretation to a decision, and an assessment validity argument, which links assessment performance to an interpretation. I then discuss ways in which issues and questions that have been raised by language testers regarding uses, abuses, consequences, validity, and fairness in language testing can provide a basis for articulating claims and counterclaims in an assessment use argument. In my view, an assessment use argument can guide the design and development of assessments and can also lead to a focused, efficient program for collecting the most critical evidence in support of the interpretations and uses for which the assessment is intended.