The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Implications of Evidence‐Centered Design for Educational Testing
Abstract
Evidence‐centered assessment design (ECD) provides language, concepts, and knowledge representations for designing and delivering educational assessments, all organized around the evidentiary argument an assessment is meant to embody. This article describes ECD in terms of layers for analyzing domains, laying out arguments, creating schemas for operational elements such as tasks and measurement models, implementing the assessment, and carrying out the operational processes. We argue that this framework helps designers take advantage of developments from measurement, technology, cognitive psychology, and learning in the domains. Examples of ECD tools and applications are drawn from the Principled Assessment Design for Inquiry (PADI) project. Attention is given to implications for large‐scale tests such as state accountability measures, with a special eye for computer‐based simulation tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Situated Cognition and the Culture of Learning
Brown, J. S., Collins, A., & Duguid, P. (1989) · Educational Researcher · doi:10.3102/0013189x018001032 · Cited by 13,645
National Science Education Standards
Shea, J. H. (1995) · Journal of Geological Education · doi:10.5408/0022-1368-43.2.102 · Cited by 7,888
Toulmin, S. E. (2003) · doi:10.1017/cbo9780511840005 · Cited by 7,101
A Pattern Language: Towns, Buildings, Construction
Alexander, C., Ishikawa, S., Silverstein, M., Jacobson, M., Fiksdahl-King, I., & Angel, S. (1977) · Cited by 6,582
Johnson-Laird, P. N. (1983) · Cited by 5,190
Artificial neural networks: a tutorial
Jain, A. K., Mao, J., & Mohiuddin, K. M. (1996) · Computer · doi:10.1109/2.485891 · Cited by 4,946
Educational measurement, 3rd ed.
Linn, R. L. (1989) · Cited by 2,883
Knowing What Students Know: The Science and Design of Educational Assessment
Pellegrino, J. W., Chudowsky, N., & Glaser, R. A. (2013) · Cited by 2,402
Kenyon, D. M., Bachman, L. F., & Palmer, A. S. (1998) · The Modern Language Journal · doi:10.2307/328718 · Cited by 2,323
Douglas, O. B., & Holland, B. F. (2015) · Fundamentals of educational psychology. · doi:10.1037/14910-020 · Cited by 2,272
Handbook of research on science teaching and learning
(1995) · Choice Reviews Online · doi:10.5860/choice.32-4019 · Cited by 2,062
Lasater, C. G. (2006) · Cited by 1,682
Inquiry, modeling, and metacognition: making science accessible to all students
White, B. Y., & Frederiksen, J. R. (1998) · Cognition and Instruction · doi:10.1207/s1532690xci1601_2 · Cited by 1,394 · cited 1×
“PADI's applications in science revolve around paradigmatic ways of thinking, such as inquiry cycles (White & Frederiksen, 1998), knowledge representation (Markman, 1999; Greeno, 1983), model-based reasoning (Stewart & Hafner, 1994), and scaffolded learning (Brown, Collins, & Duguid, 1989).”
BackgroundHow Buildings Learn: What Happens After They're Built
Brand, S. (1997) · Cited by 1,320
The interplay of evidence and consequences in the validation of performance assessments
Messick, S. (1994) · Educational Researcher · doi:10.3102/0013189x023002013 · Cited by 1,315
(2004) · Cited by 1,236
The Multidimensional Random Coefficients Multinomial Logit Model
Adams, R. J., Wilson, M., & Wang, W.-C. (1997) · Applied Psychological Measurement · doi:10.1177/0146621697211001 · Cited by 943
An argument-based approach to validity.
Kane, M. T. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.112.3.527 · Cited by 901
Focus Article: On the Structure of Educational Assessments
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003) · Measurement: Interdisciplinary Research & Perspective · doi:10.1207/s15366359mea0101_02 · Cited by 802
Self-citationNational Science Education Standards
Lagowski, J. J. (1995) · Journal of Chemical Education · doi:10.1021/ed072p287 · Cited by 770
Lane, S. (2015) · Routledge eBooks · doi:10.4324/9780203102961 · Cited by 728
Marshall, S. P. (1995) · Cambridge University Press eBooks · doi:10.1017/cbo9780511527890 · Cited by 659
Argument-Based Approach to Validity
(2022) · The SAGE Encyclopedia of Research Design · doi:10.4135/9781071812082.n28 · Cited by 556
The Acquisition of Expert Performance: An Introduction to Some of the Issues
Ericsson, K. A. (1996) · doi:10.4324/9781315805948-6 · Cited by 481
Epistemic forms and Epistemic Games: Structures and Strategies to Guide Inquiry
Collins, A., & Ferguson, W. (1993) · Educational Psychologist · doi:10.1207/s15326985ep2801_3 · Cited by 367
The Argument-Based Approach to Validation
Kane, M. (2013) · School Psychology Review · doi:10.1080/02796015.2013.12087465 · Cited by 356
A cognitive design system approach to generating valid tests: Application to abstract reasoning.
Embretson, S. E. (1998) · Psychological Methods · doi:10.1037//1082-989x.3.3.380 · Cited by 349
Item Generation for Test Development
(2013) · Routledge eBooks · doi:10.4324/9781410602145 · Cited by 298
Mind Bugs: The Origins of Procedural Misconceptions
VanLehn, K. A. (1990) · Cited by 289
Markman, A. B. (2013) · Psychology Press eBooks · doi:10.4324/9780203763698 · Cited by 273
A GENERAL DIAGNOSTIC MODEL APPLIED TO LANGUAGE TESTING DATA
von Davier, M. (2005) · ETS Research Report Series · doi:10.1002/j.2333-8504.2005.tb01993.x · Cited by 259
A General Latent Trait Model for Response Processes
Embretson (Whitely), S. (1984) · Psychometrika · doi:10.1007/bf02294171 · Cited by 235
Automated Scoring of Complex Tasks in Computer-Based Testing
(2006) · Routledge eBooks · doi:10.4324/9780415963572 · Cited by 227
On the theory of achievement test items
Bormuth, J. R. (1970) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 175
Cultures de production, de distribution et de consommation en milieu urbain en Angleterre, 1100-1350
Keene, D. (2006) · Histoire urbaine · doi:10.3917/rhu.016.0017 · Cited by 170
Shanahan, J. G. (2000) · Soft Computing for Knowledge Discovery · doi:10.1007/978-1-4615-4335-0_2 · Cited by 167
Enhancing the Design and Delivery of Assessment Systems: A Four-Process Architecture
Almond, R. G., Steinberg, L. S., Mislevy, R. J., & Hoffmann, T. (2002) · Open Access Journals at BC (Boston College) · Cited by 144
Self-citationMaking Sense of Data From Complex Assessments
Mislevy, R. J., Steinberg, L. S., Breyer, F., Almond, R. G., & Johnson, L. A. (2002) · Applied Measurement in Education · doi:10.1207/s15324818ame1504_03 · Cited by 109
Self-citationInnovative Simulations for Assessing Professional Competence
McGaghie, W. C., Tekian, A. S., & Ch, M. (1999) · Cited by 83
Assessment of Problem Solving Using Simulations
(2017) · Routledge eBooks · doi:10.4324/9781315096773 · Cited by 81
Behrens, J. T., Mislevy, R. J., Bauer, M. I., et al. (2004) · International Journal of Testing · doi:10.1207/s15327574ijt0404_1 · Cited by 62
Self-citationHafner, R., & Stewart, J. (1995) · Science Education · doi:10.1002/sce.3730790202 · Cited by 56
Accessibility of tests for individuals with disabilities within a validity framework
Hansen, E. G., Mislevy, R. J., Steinberg, L. S., Lee, M. J., & Forer, D. C. (2005) · System · doi:10.1016/j.system.2004.11.002 · Cited by 34
Self-citationRupp, A. A. (2002) · International Journal of Testing · doi:10.1080/15305058.2002.9669498 · Cited by 22
The Case for an Integrated Design Framework for Assessing Science Inquiry
Baxter, G. P., & Mislevy, R. J. (2004) · Cited by 9
Self-citation
Research works printed but not linked: 22
- American Association for the Advancement of Science (AAAS), 1993, Benchmarks for scientific literacy
- 10.1080/00405849709543775 · doi:10.1080/00405849709543775
- Baker E. L.(2002).Design of automated authoring systems for tests.Proceedings of technology and assessment: Thinking ahead proceedings from a workshop(pp.79–89). Washington DC: National Research Council Board on Testing and Assessment Center for Education Division of Behavioral and Social Sciences and Education.
- Chung G. K. Delacruz W. K. Dionne G. B. &Bewley W. L.(2003 December).Linking assessment and instruction using ontologies.Proceedings of the I/ITSEC Orlando FL .
- Cisco Systems, 2000, Internetworking technology basics
- 10.1037/1082-989X.3.3.380 · doi:10.1037/1082-989x.3.3.380 · Paper page
- Ericsson K. A., 1996, The road to excellence: The acquisition of expert performances, sports, and games
- Gotwals A. W., 2006, Cognitive predictions: BioKIDS implementation of the PADI assessment system
- Hamel L., 2006, A Wizard for PADI assessment design
- IMS Global Learning Consortium(2000).IMS question and test interoperability specification: A review(White Paper IMSWP‐1 Version A). Burlington MA :IMS Global Learning Consortium.
- Messick S., 1989, Educational measurement, 13
- Niemi D., 2005, Moving technology up‐design requirements for valid, effective classroom and large‐scale assessment
- Niemi D., 2005, Moving technology up‐design requirements for valid, effective classroom and large‐scale assessment
- Pearlman M.(2001).Performance assessments for adult education: How to design performance tasks. Presented at the workshop“Performance Assessments for Adult Education: Exploring the Measurement Issues ” hosted by the National Research Council's Board on Testing and Assessment December 12–13 Washington DC .
- 10.1207/S15327574IJT023&4_7 · doi:10.1207/s15327574ijt023&4_7
- Scalise K., 2003, Innovative item types and outcome spaces in computer‐adaptive assessment: A literature survey
- Shute V. J., 2000, Computers as Cognitive Tools, Volume 2, 309
- Simon H. A., 2001, The sciences of the artificial
- Songer N. B.(2004)Evidence of complex reasoning in technology and science: Notes from Inner City Detroit Michigan USA. IPSI‐2004 Pescara Conference Italy .
- Stevens R., 2006, Automated scoring of complex tasks in computer based testing, 259
- Webb N., 2006, Handbook of test development, 155
- Wickham D., 2002, Designing effective wizards: A multidisciplinary approach
Cited by
Foundations of Game-Based Learning
Plass, J. L., Homer, B. D., & Kinzer, C. K. (2015) · Educational Psychologist · doi:10.1080/00461520.2015.1122533 · Cited by 1,697
PISA 2018 Assessment and Analytical Framework
OECD (2019) · PISA · doi:10.1787/b25efab8-en · Cited by 1,614
VLAT: Development of a Visualization Literacy Assessment Test
Lee, S., Kim, S.-H., & Kwon, B. C. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2598920 · Cited by 290
The development and validation of a learning progression for argumentation in science
Osborne, J. F., Henderson, J. B., MacPherson, A., Szu, E., Wild, A., & Yao, S. (2016) · Journal of Research in Science Teaching · doi:10.1002/tea.21316 · Cited by 277
Siddiq, F., Hatlevik, O. E., Olsen, R. V., Throndsen, I., & Scherer, R. (2016) · Educational Research Review · doi:10.1016/j.edurev.2016.05.002 · Cited by 269
Metadata
- Authors
- Mislevy, R. J., & Haertel, G. D.
- Venue
- Educational Measurement: Issues and Practice
- Publisher
- Wiley
- Published
- 2006-12
- Volume
- 25
- Issue
- 4
- Pages
- 6-20
- ISSN
- 0731-1745 (ISSN-L)
- 1745-3992
- OpenAlex
- W2063747225
- Cited by
- 543
- References
- 45
- FWCI
- 5.28
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.