The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Focus Article: On the Structure of Educational Assessments
Abstract
In educational assessment, we observe what students say, do, or make in a few particular circumstances and attempt to infer what they know, can do, or have accomplished more generally. A web of inference connects the two. Some connections depend on theories and experience concerning the targeted knowledge in the domain, how it is acquired, and the circumstances under which people bring their knowledge to bear. Other connections may depend on statistical models and probability-based reasoning. Still others concern the elements and processes involved in test construction, administration, scoring, and reporting. This article describes a framework for assessment that makes explicit the interrelations among substantive arguments, assessment designs, and operational processes. The work was motivated by the need to develop assessments that incorporate purposes, technologies, and psychological perspectives that are not well served by familiar forms of assessments. However, the framework is equally applicable to analyzing existing assessments or designing new assessments within familiar forms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195 · cited 1×
“…and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment” (Messick, 1989, p. 13; see Cronbach, 1989; Cronbach & Meehl, 1955; Embretson, 1983, 1998; Kane, 1992; Messick, 1989, 1994; Sarbin, Taft, & Bailey, 1960).”
BackgroundToulmin, S. E. (2003) · doi:10.1017/cbo9780511840005 · Cited by 7,101
Probabilistic Models for Some Intelligence and Attainment Tests
(1961) · Information and Control · doi:10.1016/s0019-9958(61)80061-2 · Cited by 6,939
Habermas, J. (1968) · Cited by 4,683
Fundamental Considerations in Language Testing
Spolsky, B., & Bachman, L. F. (1991) · The Modern Language Journal · doi:10.2307/329499 · Cited by 4,362
Handbook of Educational Psychology
(2013) · Routledge eBooks · doi:10.4324/9780203053874 · Cited by 3,212
Educational measurement, 3rd ed.
Linn, R. L. (1989) · Cited by 2,883
Knowledge and Human Interests.
Parsons, H. L., Habermas, J., & Shapiro, J. J. (1972) · Philosophy and Phenomenological Research · doi:10.2307/2106475 · Cited by 2,606
(2007) · Encyclopedia of Measurement and Statistics · doi:10.4135/9781412952644.n300 · Cited by 2,578
Knowing What Students Know: The Science and Design of Educational Assessment
Pellegrino, J. W., Chudowsky, N., & Glaser, R. A. (2013) · Cited by 2,402
Kenyon, D. M., Bachman, L. F., & Palmer, A. S. (1998) · The Modern Language Journal · doi:10.2307/328718 · Cited by 2,323
The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles
Merrifield, P. R., Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1974) · American Educational Research Journal · doi:10.2307/1162145 · Cited by 1,819
The interplay of evidence and consequences in the validation of performance assessments
Messick, S. (1994) · Educational Researcher · doi:10.3102/0013189x023002013 · Cited by 1,315
Measuring Second Language Performance
Kleinsasser, R. C., & McNamara, T. (1998) · The Modern Language Journal · doi:10.2307/330236 · Cited by 1,253
Fundamental considerations in language testing
Bachman, L. F. (1991) · Cited by 1,236
(2013) · Human Behavior · doi:10.1002/9781394260546.ch8 · Cited by 1,215
The Multidimensional Random Coefficients Multinomial Logit Model
Adams, R. J., Wilson, M., & Wang, W.-C. (1997) · Applied Psychological Measurement · doi:10.1177/0146621697211001 · Cited by 943
Educative assessment: designing assessments to inform and improve student performance
(1999) · Choice Reviews Online · doi:10.5860/choice.36-2887 · Cited by 931
Bayesian Analysis in Expert Systems
Spiegelhalter, D. J., Dawid, A. P., Lauritzen, S. L., & Cowell, R. G. (1993) · Statistical Science · doi:10.1214/ss/1177010888 · Cited by 684
The linear logistic test model as an instrument in educational research
Fischer, G. H. (1973) · Acta Psychologica · doi:10.1016/0001-6918(73)90003-6 · Cited by 645
Construct validity: Construct representation versus nomothetic span.
Whitely, S. E. (1983) · Psychological Bulletin · doi:10.1037//0033-2909.93.1.179 · Cited by 604
Elements of Generalizability Theory
Brennan, R. L. (1992) · Medical Entomology and Zoology · Cited by 599
Laser cooling and trapping of neutral atoms
Adams, C. S., & Riis, E. (1997) · Progress in Quantum Electronics · doi:10.1016/s0079-6727(96)00006-7 · Cited by 557
Can There Be Validity without Reliability?
Moss, P. A. (1994) · Educational Researcher · doi:10.2307/1176218 · Cited by 556
Argument-Based Approach to Validity
(2022) · The SAGE Encyclopedia of Research Design · doi:10.4135/9781071812082.n28 · Cited by 556
Cognitive modeling and intelligent tutoring
Anderson, J. R., Boyle, C. F., Corbett, A. T., & Lewis, M. W. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90093-f · Cited by 534
The Evidential Foundations of Probabilistic Reasoning.
FH, & Schum, D. A. (1995) · Journal of the American Statistical Association · doi:10.2307/2291371 · Cited by 506
Cognitively Diagnostic Assessment
(2012) · Routledge eBooks · doi:10.4324/9780203052969 · Cited by 490
Diagnostic Monitoring of Skill and Knowledge Acquisition
(2013) · Routledge eBooks · doi:10.4324/9780203056899 · Cited by 468
Objective measurement : theory into practice
Wilson, M. R., Engelhard, G., & Draney, K. (1992) · Ablex Pub. Corp. eBooks · Cited by 454
The Difficulty of Test Items That Measure More Than One Ability
Reckase, M. D. (1985) · Applied Psychological Measurement · doi:10.1177/014662168500900409 · Cited by 424
A cognitive design system approach to generating valid tests: Application to abstract reasoning.
Embretson, S. E. (1998) · Psychological Methods · doi:10.1037//1082-989x.3.3.380 · Cited by 349
Applications and Extensions of MCMC in IRT: Multiple Item Types, Missing Data, and Rated Responses
Patz, R. J., & Junker, B. W. (1999) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165367 · Cited by 345
A Bayesian Random Effects Model for Testlets
Bradlow, E. T., Wainer, H., & Wang, X. (1999) · Psychometrika · doi:10.1007/bf02294533 · Cited by 342
CONSTRUCT VALIDITY: CONSTRUCT REPRESENTATION VERSUS NOMOTHETIC SPAN
Embretson, S. E. (1983) · Cited by 322
A BAYESIAN RANDOM EFFECTS MODEL FOR TESTLETS
Bradlow, E. T., Wainer, H., & Wang, X. (1998) · ETS Research Report Series · doi:10.1002/j.2333-8504.1998.tb01752.x · Cited by 315
Construct validation after thirty years.
Cronbach, L. J. (1989) · Cited by 277
Literacy, profiles of America's young adults
Kirsch, I. S., & Jungeblut, A. (1986) · Cited by 241
Toward an Integration of IteD1-Response Theory and Cognitive Error Diagnosis
(2013) · Diagnostic Monitoring of Skill and Knowledge Acquisition · doi:10.4324/9780203056899-22 · Cited by 215
EVIDENCE AND INFERENCE IN EDUCATIONAL ASSESSMENT1,2
Mislevy, R. J. (1995) · ETS Research Report Series · doi:10.1002/j.2333-8504.1995.tb01643.x · Cited by 197
Self-citationEvidence and Inference in Educational Assessment
Mislevy, R. J. (1994) · Psychometrika · doi:10.1007/bf02294388 · Cited by 165
Self-citationThe role of probability-based inference in an intelligent tutoring system
Mislevy, R. J., & Gitomer, D. H. (1996) · User Modeling and User-Adapted Interaction · doi:10.1007/bf01126112 · Cited by 163
Self-citationChapter 2: To Use Their Minds Well: Investigating New Forms of Student Assessment
Wolf, D., Bixby, J., Glenn, J., & Gardner, H. (1991) · Review of Research in Education · doi:10.3102/0091732x017001031 · Cited by 161
Clinical inference and cognitive theory
Sarbin, T. R., Taft, R., & Bailey, D. E. (1960) · Medical Entomology and Zoology · Cited by 160
GRAPHICAL MODELS AND COMPUTERIZED ADAPTIVE TESTING
Almond, R. G., & Mislevy, R. J. (1998) · ETS Research Report Series · doi:10.1002/j.2333-8504.1998.tb01753.x · Cited by 122
Self-citationBayes nets in educational assessment: Where the numbers come fromPreprint
Mislevy, R. J., Almond, R. G., Yan, D., & Steinberg, L. S. (1999) · arXiv (Cornell University) · Cited by 118
Self-citationGraphical Models and Computerized Adaptive Testing
Almond, R. G., & Mislevy, R. J. (1999) · Applied Psychological Measurement · doi:10.1177/01466219922031347 · Cited by 117
Self-citationEvidence and inference for the intelligence analyst
Schum, D. A. (1987) · Cited by 110
Making Sense of Data From Complex Assessments
Mislevy, R. J., Steinberg, L. S., Breyer, F., Almond, R. G., & Johnson, L. A. (2002) · Applied Measurement in Education · doi:10.1207/s15324818ame1504_03 · Cited by 109
Self-citationThe evidential foundations of probabilistic reasoning
Schum, D. A. (1994) · Cited by 97
Green, B. F. (1978) · American Psychologist · doi:10.1037/0003-066x.33.7.664 · Cited by 76
A theory of the measurement of knowledge content, access, and learning.
Pirolli, P. L. T., & Wilson, M. R. (1998) · Psychological Review · doi:10.1037/0033-295x.105.1.58 · Cited by 75
On the Roles of Task Model Variables in Assessment Design
Kyllonen, P. C. (2013) · Routledge eBooks · doi:10.4324/9781410602145-12 · Cited by 66
An argument-based approach to validity.
Kane, M. T. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.3.527 · Cited by 66
Monitoring and Improving a Portfolio Assessment System.
Myford, C. M., & Mislevy, R. J. (1995) · Cited by 60
Self-citationA Sample Assessment Using the Four Process Framework. CSE Technical Report 543.
Almond, R. G., Steinberg, L. S., & Mislevy, R. J. (2001) · Cited by 11
Self-citation
Cited by
Validating the interpretations and uses of test scores
Kane, M. T. (2013) · Journal of Educational Measurement · doi:10.1111/jedm.12000 · Cited by 2,110
What Is Design Thinking and Why Is It Important?
Razzouk, R., & Shute, V. (2012) · Review of Educational Research · doi:10.3102/0034654312457429 · Cited by 1,388
Language testing: the social dimension
Roever, C., & McNamara, T. (2006) · International Journal of Applied Linguistics · doi:10.1111/j.1473-4192.2006.00117.x · Cited by 668
Implications of Evidence‐Centered Design for Educational Testing
Mislevy, R. J., & Haertel, G. D. (2006) · Educational Measurement: Issues and Practice · doi:10.1111/j.1745-3992.2006.00075.x · Cited by 543
Self-citationA BRIEF INTRODUCTION TO EVIDENCE‐CENTERED DESIGN
Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003) · ETS Research Report Series · doi:10.1002/j.2333-8504.2003.tb01908.x · Cited by 473
Self-citation
Metadata
- Authors
- Mislevy, R. J., Steinberg, L. S., & Almond, R. G.
- Venue
- Measurement: Interdisciplinary Research & Perspective
- Publisher
- Informa UK Limited
- Published
- 2003-01
- Volume
- 1
- Issue
- 1
- Pages
- 3-62
- ISSN
- 1536-6359 (ISSN-L)
- 1536-6367
- OpenAlex
- W2039681496
- Cited by
- 802
- References
- 57
- FWCI
- 125.68
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.