The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Can There Be Validity without Reliability?
Abstract
Reliability has traditionally been taken for granted as a necessary but insufficient condition for validity in assessment use. My purpose in this article is to illuminate and challenge this presumption by exploring a dialectic between psychometric and hermeneutic approaches to drawing and warranting interpretations of human products or performances. Reliability, as it is typically defined and operationalized in the measurement literature (e.g., American Educational Research Association [AERA], American Psychological Association, & National Council on Measurement in Education, 1985 ; Feldt & Brennan, 1989 ), privileges standardized forms of assessment. By considering hermeneutic alternatives for serving the important epistemological and ethical purposes that reliability serves, we expand the range of viable high-stakes assessment practices to include those that honor the purposes that students bring to their work and the contextualized judgments of teachers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Handbook of research on teaching
Wittrock, M. C. (1986) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 10,626
Introduction to Classical and Modern Test Theory
Crocker, L. M., & Algina, J. J. (1986) · Cited by 5,090
Essentials of Psychological Testing
Rosner, B., & Cronbach, L. J. (1960) · The American Journal of Psychology · doi:10.2307/1419921 · Cited by 4,513
Educational measurement, 3rd ed.
Linn, R. L. (1989) · Cited by 2,882
Hermeneutics and the Human Sciences: Essays on Language, Action and Interpretation
Ricoeur, P. (2016) · Cambridge University Press eBooks · doi:10.1017/cbo9781316534984 · Cited by 2,651
Beyond Objectivism and Relativism: Science, Hermeneutics, and Praxis
Bernstein, R. J. (1984) · University of Pennsylvania Press eBooks · doi:10.9783/9780812205503 · Cited by 2,166
Essentials of Psychological Testing
Urbina, S. (2014) · doi:10.1002/9781394259458 · Cited by 1,845
Humphreys, L. G., & Lindquist, E. F. (1952) · The American Journal of Psychology · doi:10.2307/1418848 · Cited by 1,502
Standards for educational and psychological testing: (500832009-001)
(1985) · PsycEXTRA Dataset · doi:10.1037/e500832009-001 · Cited by 1,480
The interplay of evidence and consequences in the validation of performance assessments
Messick, S. (1994) · Educational Researcher · doi:10.3102/0013189x023002013 · Cited by 1,315
The essential tension : selected studies in scientific tradition and change
Kühn, T. (1977) · Philosophy of Science · doi:10.7208/9780226217239 · Cited by 1,198
Beyond objectivism and relativism
Bernstein, R. J. (1983) · Cited by 1,096
Validation in Inquiry-Guided Research: The Role of Exemplars in Narrative Studies
Mishler, E. (1990) · Harvard Educational Review · doi:10.17763/haer.60.4.n4405243p6635752 · Cited by 1,004
The model of the text: meaningful action considered as a text
(2016) · Hermeneutics and the Human Sciences · doi:10.1017/cbo9781316534984.011 · Cited by 970
Complex, Performance-Based Assessment: Expectations and Validation Criteria
Linn, R. L., Baker, E. L., & Dunbar, S. B. (1991) · Educational Researcher · doi:10.2307/1176232 · Cited by 926
Test validity and the ethics of assessment.
Messick, S. (1980) · American Psychologist · doi:10.1037/0003-066x.35.11.1012 · Cited by 862
Gadamer, Hermeneutics, Tradition and Reason
Flanagan, K., & Warnke, G. (1989) · British Journal of Sociology · doi:10.2307/590284 · Cited by 836
Qualitative Inquiry in Education: The Continuing Debate
Stokrocki, M., Eisner, E., & Peshkin, A. (1993) · Studies in Art Education · doi:10.2307/1320628 · Cited by 810
A Systems Approach to Educational Testing
Frederiksen, J. R., & Collins, A. (1989) · Educational Researcher · doi:10.2307/1176716 · Cited by 648
Assessing the Thinking Curriculum: New Tools for Educational Reform
Resnick, L. B., & Resnick, D. P. (1992) · Changing Assessments · doi:10.1007/978-94-011-2968-8_3 · Cited by 589
Interpretive social science : a second look
Rabinow, P., & Sullivan, W. M. (1987) · Cited by 527
Entering the circle: hermeneutic investigation in psychology
(1989) · Choice Reviews Online · doi:10.5860/choice.27-2381 · Cited by 520
Put to the Test: The Effects of External Testing on Teachers
Smith, M. L. (1991) · Educational Researcher · doi:10.2307/1176396 · Cited by 509
Shifting Conceptions of Validity in Educational Measurement: Implications for Performance Assessment
Moss, P. A. (1992) · Review of Educational Research · doi:10.2307/1170738 · Cited by 434
Self-citationThe Essential Tension: Selected Studies in Scientific Tradition and Change
Hollinger, D. A., & Kuhn, T. S. (1978) · The American Historical Review · doi:10.2307/1854704 · Cited by 433
The standard problem: Meaning and values in measurement and evaluation.
Messick, S. (1975) · American Psychologist · doi:10.1037/0003-066x.30.10.955 · Cited by 417
Sampling Variability of Performance Assessments
Shavelson, R. J., Baxter, G. P., & Gao, X. (1993) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1993.tb00424.x · Cited by 338
Contemporary Hermeneutics: Hermeneutics as Method, Philosophy and Critique.
Bubner, R., & Bleicher, J. (1983) · The Philosophical Review · doi:10.2307/2184500 · Cited by 290
Newmann, F. M. (1990) · Journal of Curriculum Studies · doi:10.1080/0022027900220103 · Cited by 285
Construct validation after thirty years.
Cronbach, L. J. (1989) · Cited by 277
Quality Control in the Development and Use of Performance Assessments
Dunbar, S. B., Koretz, D. M., & Hoover, H. D. (1991) · Applied Measurement in Education · doi:10.1207/s15324818ame0404_3 · Cited by 274
Beyond Objectivism and Relativism: Science, Hermeneutics and Praxis
Harrison, K. (1985) · Telos · doi:10.3817/0385063223 · Cited by 236
Educational Measurement: Issues and Practice
(2010) · Educational Measurement Issues and Practice · doi:10.1111/(issn)1745-3992 · Cited by 227
Contemporary Hermeneutics: Hermeneutics as Method, Philosophy and Critique
Rickman, H. P., & Bleicher, J. (1982) · The British Journal of Sociology · doi:10.2307/589940 · Cited by 208
Accountability for Professional Practice
Darling-Hammond, L. (1989) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1177/016146818909100108 · Cited by 161
Chapter 2: To Use Their Minds Well: Investigating New Forms of Student Assessment
Wolf, D., Bixby, J., Glenn, J., & Gardner, H. (1991) · Review of Research in Education · doi:10.3102/0091732x017001031 · Cited by 161
How Does Social Science Work? Reflections on Practice
Diesing, P. (1992) · University of Pittsburgh Press eBooks · doi:10.2307/j.ctt5hjpmm · Cited by 158
Perrone, V. (1991) · Cited by 156
Using Performance Assessment for Accountability Purposes
Mehrens, W. A. (1992) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.1992.tb00220.x · Cited by 151
The Meaning of Scholarly Activity and the Building of Community
Lieberman, A. (1992) · Educational Researcher · doi:10.3102/0013189x021006005 · Cited by 136
Cole, N. S., & Moss, P. (1989) · Cited by 123
Self-citationToward a new science of educational testing and assessment
(1992) · Choice Reviews Online · doi:10.5860/choice.30-0434 · Cited by 116
Philosophical redirection of educational research
Thomas, L. G. (1972) · National Society for the Study of Education , Distributed by the University of Chicago Press eBooks · Cited by 66
Addressing Reliability Problems in the Portfolio Assessment of College Writing
Nystrand, M., Cohen, A. S., & Dowling, N. M. (1993) · Educational Assessment · doi:10.1207/s15326977ea0101_4 · Cited by 60
Portfolios, Accountability, and an Interpretive Approach to Validity
Moss, P., Beck, J. S., Ebbs, C. A., Matson, B. L. S., Muchmore, J. A., Steele, D. D., Taylor, C., & Herter, R. J. (1992) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.1992.tb00244.x · Cited by 46
Self-citationTHE INTERPLAY OF EVIDENCE AND CONSEQUENCES IN THE VALIDATION OF PERFORMANCE ASSESSMENTS
Messick, S. (1992) · ETS Research Report Series · doi:10.1002/j.2333-8504.1992.tb01470.x · Cited by 42
Scriven, M. (1972) · Yearbook of the National Society for the Study of Education · Cited by 19
Cited by
Validating the interpretations and uses of test scores
Kane, M. T. (2013) · Journal of Educational Measurement · doi:10.1111/jedm.12000 · Cited by 2,110
Evaluating Interpretive Inquiry: Reviewing the Validity Debate and Opening the Dialogue
Angen, M. J. (2000) · Qualitative Health Research · doi:10.1177/104973200129118516 · Cited by 998
Coloring Epistemologies: Are Our Research Epistemologies Racially Biased?
Scheurich, J. J., & Young, M. D. (1997) · Educational Researcher · doi:10.2307/1176879 · Cited by 842
Focus Article: On the Structure of Educational Assessments
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003) · Measurement: Interdisciplinary Research & Perspective · doi:10.1207/s15366359mea0101_02 · Cited by 802
Complementary and Alternative Medicine Use in Australia: A National Population-Based Survey
Xue, C. C. L., Zhang, A. L., Lin, V., Da Costa, C., & Story, D. F. (2007) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2006.6355 · Cited by 645
Metadata
- Authors
- Moss, P. A.
- Venue
- Educational Researcher
- Published
- 1994-03
- Volume
- 23
- Issue
- 2
- Pages
- 5
- ISSN
- 0013-189X (ISSN-L)
- OpenAlex
- W2057671577
- Cited by
- 556
- References
- 47
- FWCI
- 139.24
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.