The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Shifting Conceptions of Validity in Educational Measurement: Implications for Performance Assessment
Abstract
Recent developments in the philosophy of validity, highlighting the importance of investigating the consequences of assessment use, provide theoretical support for the move toward performance assessment. The problem for validity researchers is finding the appropriate set of criteria and standards to simultaneously support the validity of an assessment-based interpretation and the validity of its impact on the educational system. My intent is to provide an integrative and critical review of the guidance available for conducting validity inquiry in the context of performance assessment. In the first section is a summary of the emerging consensus among measurement scholars—not reflected in the 1985 Standards for Educational and Psychological Testing (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME])—about the centrality of construct validity to the evaluation of any assessment-based interpretation and about the importance of expanding the concept of validity to include explicit consideration of the consequences of assessment use. The description of this emerging consensus suggests general epistemological principles for validity inquiry. In the second section is a description and synthesis of various categories of questions, evidence, or criteria that have been used to describe or guide validity inquiry, either for assessment in general or for performance assessment in particular. These analytic schemes, like the traditional construct-content-criterion categories, highlight specific issues that their authors consider important for validity researchers to address. Each balances technical concerns about such issues as reliability, generalizability, and comparability with concerns about the consequences of assessment. In the final section is an overview of concerns, expressed largely by interpretive researchers, about validity criteria that privilege standardized forms of assessment, whether performance-based or multiple-choice. These arguments suggest the importance of further expanding the conception of validity inquiry to treat as problematic the epistemological principles used to warrant validity conclusions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lincoln, Y. S., Guba, E. G., & Pilotta, J. J. (1985) · International Journal of Intercultural Relations · doi:10.1016/0147-1767(85)90062-8 · Cited by 45,787
Basics of Qualitative Research
Ozanne, J. L., Strauss, A., & Corbin, J. (1992) · Journal of Marketing Research · doi:10.2307/3172751 · Cited by 17,850
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Qualitative Research for Education: An Introduction to Theory and Methods
Bogdan, R., & Biklen, S. K. (1997) · Medical Entomology and Zoology · Cited by 15,145
(1963) · Medical Entomology and Zoology · Cited by 15,085
(1979) · Medical Entomology and Zoology · Cited by 11,252
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
(1990) · Cited by 11,110
Handbook of research on teaching
Wittrock, M. C. (1986) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 10,626
(2014) · The SAGE Encyclopedia of Action Research · doi:10.4135/9781446294406.n148 · Cited by 7,250
The Research Act: A Theoretical Introduction to Sociological Methods
Fox, W. S., & Denzin, N. K. (1979) · Contemporary Sociology · doi:10.2307/2065439 · Cited by 6,284
Neufeld, A., & Ryan, A. G. (1988) · Canadian Journal of Program Evaluation · doi:10.3138/cjpe.3.006 · Cited by 5,913
Case Study Research in Education : A Qualitative Approach
Merriam, S. B. (1991) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 5,699
Essentials of Psychological Testing
Rosner, B., & Cronbach, L. J. (1960) · The American Journal of Psychology · doi:10.2307/1419921 · Cited by 4,513
Ethnography and Qualitative Design in Educational Research
LeCompte, M. D., Preissle, J., & Tesch, R. (1984) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 4,261
Saeverot, H. (2022) · Indirect Education · doi:10.4324/9781003193463-8 · Cited by 3,059
Noblit, G., & Hare, R. (1988) · doi:10.4135/9781412985000 · Cited by 2,723
The Research Act: A Theoretical Introduction to Sociological Methods
Krause, D., & Denzin, N. K. (1989) · Teaching Sociology · doi:10.2307/1318434 · Cited by 2,564
Douglas, O. B., & Holland, B. F. (2015) · Fundamentals of educational psychology. · doi:10.1037/14910-020 · Cited by 2,272
Meta-Ethnography: Synthesizing Qualitative Studies
Noblit, G. W., & Hare, R. D. (1988) · Medical Entomology and Zoology · Cited by 2,191
Beyond the two disciplines of scientific psychology.
Cronbach, L. J. (1975) · American Psychologist · doi:10.1037/h0076829 · Cited by 2,094
Logic of Scientific Discovery, The
(2010) · Encyclopedia of Research Design · doi:10.4135/9781412961288.n223 · Cited by 1,545
Zinkus, P. W. (1987) · Textbook of Developmental Pediatrics · doi:10.1007/978-1-4613-1797-5_8 · Cited by 1,500
Essentials of Educational Measurement
Garvin, A. D., & Ebel, R. L. (1980) · Educational Researcher · doi:10.2307/1175572 · Cited by 1,481
Standards for educational and psychological testing: (500832009-001)
(1985) · PsycEXTRA Dataset · doi:10.1037/e500832009-001 · Cited by 1,480
Objective Tests as Instruments of Psychological Theory
Loevinger, J. (1957) · Psychological Reports · doi:10.2466/pr0.1957.3.3.635 · Cited by 1,473
Measurement and evaluation in teaching
Gronlund, N. E. (1965) · Cited by 1,425
(2018) · The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation · doi:10.4135/9781506326139.n223 · Cited by 1,384
Factors relevant to the validity of experiments in social settings.
Campbell, D. T. (1957) · Psychological Bulletin · doi:10.1037/h0040950 · Cited by 1,140
Meaning and Values in Test Validation: The Science and Ethics of Assessment
Messick, S. (1989) · Educational Researcher · doi:10.3102/0013189x018002005 · Cited by 948
Complex, Performance-Based Assessment: Expectations and Validation Criteria
Linn, R. L., Baker, E. L., & Dunbar, S. B. (1991) · Educational Researcher · doi:10.2307/1176232 · Cited by 926
Test validity and the ethics of assessment.
Messick, S. (1980) · American Psychologist · doi:10.1037/0003-066x.35.11.1012 · Cited by 862
The Structure of Scientific Theories
D., C. P., & Suppe, F. (1978) · The American Journal of Psychology · doi:10.2307/1421551 · Cited by 709
The Logic of Scientific Discovery
Jeffrey, R. C., & Popper, K. R. (1960) · Econometrica · doi:10.2307/1907578 · Cited by 674
A Systems Approach to Educational Testing
Frederiksen, J. R., & Collins, A. (1989) · Educational Researcher · doi:10.2307/1176716 · Cited by 648
The Restructuring of Social and Political Theory
Johnson, K., & Bernstein, R. J. (1977) · The Western Political Quarterly · doi:10.2307/447946 · Cited by 576
Educational and Psychological Measurement and Evaluation
Hopkins, K. D., Stanley, J. C., & Hopkins, B. R. (1972) · Medical Entomology and Zoology · Cited by 565
Recommendations for APA test standards regarding construct, trait, or discriminant validity.
Campbell, D. T. (1960) · American Psychologist · doi:10.1037/h0048255 · Cited by 560
Entering the circle: hermeneutic investigation in psychology
(1989) · Choice Reviews Online · doi:10.5860/choice.27-2381 · Cited by 520
Put to the Test: The Effects of External Testing on Teachers
Smith, M. L. (1991) · Educational Researcher · doi:10.2307/1176396 · Cited by 509
Measurement and Evaluation in Education and Psychology
Breny, H., & Mehrens, W. A. (1979) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1403221 · Cited by 493
Measurement and evaluation in education and psychology.
WICK, J. W. (1980) · Contemporary Psychology: A Journal of Reviews · doi:10.1037/018585 · Cited by 477
Bogdan, R. (1992) · Cited by 464
The standard problem: Meaning and values in measurement and evaluation.
Messick, S. (1975) · American Psychologist · doi:10.1037/0003-066x.30.10.955 · Cited by 417
Wolfle, D. (1964) · Science · doi:10.1126/science.143.3608.767 · Cited by 406
Principles of Educational and Psychological Measurement and Evaluation
Sax, G. (2019) · Medical Entomology and Zoology · Cited by 350
Technical recommendations for psychological tests and diagnostic techniques.
American Psychological Association (1954) · Psychological Bulletin · doi:10.1037/h0053479 · Cited by 322
Newmann, F. M. (1990) · Journal of Curriculum Studies · doi:10.1080/0022027900220103 · Cited by 285
Review of The structure of scientific theories. 2nd ed.
KRANTZ, D. L. (1978) · Contemporary Psychology: A Journal of Reviews · doi:10.1037/017307 · Cited by 280
Construct validation after thirty years.
Cronbach, L. J. (1989) · Cited by 277
Evolving Concepts of Test Validation
Anastasi, A. (1986) · Annual Review of Psychology · doi:10.1146/annurev.ps.37.020186.000245 · Cited by 267
Portfolio Assessment in the Reading Writing Classroom
Tierney, R. J. (1991) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 222
Standards for Educational and Psychological Tests and Manuals: (479862008-001)
(1966) · PsycEXTRA Dataset · doi:10.1037/e479862008-001 · Cited by 217
Portfolios: Process and Product
White, E. M., Belanoff, P., & Dickson, M. (1992) · College Composition and Communication · doi:10.2307/358657 · Cited by 213
Why We Need Better Assessments.
Shepard, L. A. (1989) · Educational leadership · Cited by 209
Knowledge and Values in Social and Educational Research
Bredo, E., & Feinberg, W. (1983) · Medical Entomology and Zoology · Cited by 205
Personnel Selection and Classification Systems
Dunnette, M. D., & Borman, W. C. (1979) · Annual Review of Psychology · doi:10.1146/annurev.ps.30.020179.002401 · Cited by 203
Content Validity—The Source of My Discontent
Guion, R. M. (1977) · Applied Psychological Measurement · doi:10.1177/014662167700100103 · Cited by 200
Meaning and Values in Test Validation: The Science and Ethics of Assessment
Messick, S. (1989) · Educational Researcher · doi:10.2307/1175249 · Cited by 200
Beyond Standardized Testing : Assessing Authentic Academic Achievement in the Secondary School /
Archbald, D., & Newmann, F. M. (1988) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 173
Accountability for Professional Practice
Darling-Hammond, L. (1989) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1177/016146818909100108 · Cited by 161
Chapter 2: To Use Their Minds Well: Investigating New Forms of Student Assessment
Wolf, D., Bixby, J., Glenn, J., & Gardner, H. (1991) · Review of Research in Education · doi:10.3102/0091732x017001031 · Cited by 161
Perrone, V. (1991) · Cited by 156
Proceedings of the 1967 Invitational Conference on Testing Problems.
C., E. L., & Bloom, B. S. (1969) · Journal of the American Statistical Association · doi:10.2307/2283759 · Cited by 144
Chapter 1: New Forms of Teacher Assessment
Haertel, E. H. (1991) · Review of Research in Education · doi:10.2307/1167328 · Cited by 136
Standards, Curriculum, and Performance: A Historical and Comparative Perspective
Resnick, D. P., & Resnick, L. Β. (1985) · Educational Researcher · doi:10.3102/0013189x014004005 · Cited by 130
The Cognitive Movement in Instruction
Wittrock, M. C. (1979) · Educational Researcher · doi:10.3102/0013189x008002005 · Cited by 130
Tenopyr, M. L. (1977) · Personnel Psychology · doi:10.1111/j.1744-6570.1977.tb02320.x · Cited by 125
Cole, N. S., & Moss, P. (1989) · Cited by 123
Self-citationPersonnel Selection and Classification
Tenopyr, M. L., & Oeltjen, P. D. (1982) · Annual Review of Psychology · doi:10.1146/annurev.ps.33.020182.003053 · Cited by 122
Toward a new science of educational testing and assessment
(1992) · Choice Reviews Online · doi:10.5860/choice.30-0434 · Cited by 116
Cole, N. S. (1981) · American Psychologist · doi:10.1037/0003-066x.36.10.1067 · Cited by 116
Proceedings of the 1968 Invitational Conference on Testing Problems.
Crow, E. L., & Hastings, J. T. (1971) · Journal of the American Statistical Association · doi:10.2307/2284887 · Cited by 103
Validity, vaudeville, and values: A short history of social concerns over standardized testing.
Haney, W. (1981) · American Psychologist · doi:10.1037/0003-066x.36.10.1021 · Cited by 77
To Capture the Ineffable: New Forms of Assessment in Higher Education
Ewell, P. T. (1991) · Review of Research in Education · doi:10.2307/1167330 · Cited by 71
Scriven, M. (1986) · Evaluation Practice · doi:10.1016/s0886-1633(86)80067-8 · Cited by 71
Open a new window: Validities and values in psychological measurement.
Guion, R. M. (1974) · American Psychologist · doi:10.1037/h0037624 · Cited by 56
Chapter 3: To Capture the Ineffable: New Forms of Assessment in Higher Education
Ewell, P. T. (1991) · Review of Research in Education · doi:10.3102/0091732x017001075 · Cited by 32
Mos, L. P. (1987) · Philosophy in Review · Cited by 19
Ability Testing in the 1980's and Beyond: Some Major Trends
Anastasi, A. (1989) · Public Personnel Management · doi:10.1177/009102608901800407 · Cited by 8
Cited by
Enhancing teaching through constructive alignment
Biggs, J. (1996) · Higher Education · doi:10.1007/bf00138871 · Cited by 4,084
Validating the interpretations and uses of test scores
Kane, M. T. (2013) · Journal of Educational Measurement · doi:10.1111/jedm.12000 · Cited by 2,110
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.1061 · Cited by 1,674
The interplay of evidence and consequences in the validation of performance assessments
Messick, S. (1994) · Educational Researcher · doi:10.3102/0013189x023002013 · Cited by 1,315
Thompson, B., & Daniel, L. G. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056002001 · Cited by 918
Metadata
- Authors
- Moss, P. A.
- Venue
- Review of Educational Research
- Publisher
- SAGE Publications
- Published
- 1992
- Volume
- 62
- Issue
- 3
- Pages
- 229
- ISSN
- 0034-6543 (ISSN-L)
- OpenAlex
- W2157540565
- Cited by
- 434
- References
- 80
- FWCI
- 34.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.