Design and Discovery in Educational Assessment: Evidence-Centered Design, Psychometrics, and Educational Data Mining
Open accessAbstract
Evidence-centered design (ECD) is a comprehensive framework for describing the conceptual, computational and inferential elements of educational assessment. It emphasizes the importance of articulating inferences one wants to make and the evidence needed to support those inferences. At first blush, ECD and educational data mining (EDM) might seem in conflict: structuring situations to evoke particular kinds of evidence, versus discovering meaningful patterns in available data. However, a dialectic between the two stances increases understanding and improves practice. We first introduce ECD and relate its elements to the broad range of digital inputs relevant to modern assessment. We then discuss the relation between EDM and psychometric activities in educational assessment. We illustrate points with examples from the Cisco Networking Academy, a global program in which information technology is taught through a blended program of face-to-face classroom instruction, an online curriculum, and online assessments.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
The interplay of evidence and consequences in the validation of performance assessments
Messick, S. (1994) · Educational Researcher · doi:10.3102/0013189x023002013 · Cited by 1,315 · see citing works
Jung, C. G. (1956) · Collected Works, Volume 14 · doi:10.1515/9781400845927 · Cited by 885 · cited 1× · see citing works
“Since its inception in 1997, the CNA has grown to reach a diverse population of about a million students each year in more than 165 countries [Murnane et al. 2002; Levy and Murnane 2004].”
Background
Flagged first, then by citations
Cited by
Sorted by citations
Assessment in the age of artificial intelligence
Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100075 · Cited by 409 · see citing works
Generative Artificial Intelligence in Education and Its Implications for Assessment
Mao, J., Chen, B., & Liu, J. C. (2024) · TechTrends · doi:10.1007/s11528-023-00911-4 · Cited by 270 · see citing works
Gobert, J. D., Sao Pedro, M., Raziuddin, J., & Baker, R. S. (2013) · Journal of the Learning Sciences · doi:10.1080/10508406.2013.837391 · Cited by 210 · see citing works
Gobert, J. D., Sao Pedro, M. A., Baker, R. S. J. D., Toto, E., & Montalvo, O. (2012) · Zenodo · doi:10.5281/zenodo.3554645 · Cited by 122 · see citing works
Grover, S., Basu, S., Bienkowski, M., Eagle, M., Diana, N., & Stamper, J. (2017) · ACM Transactions on Computing Education · doi:10.1145/3105910 · Cited by 93 · see citing works
Improving workplace-based assessment and feedback by an E-portfolio enhanced with learning analytics
van der Schaaf, M., Donkers, J., Slof, B., Moonen-van Loon, J., van Tartwijk, J., Driessen, E., Badii, A., Serban, O., & Ten Cate, O. (2017) · Educational Technology Research and Development · doi:10.1007/s11423-016-9496-8 · Cited by 82 · see citing works
- Authors
- Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R.
- Affiliations
- Educational Testing Service (US)
- Pearson (United States) (US)
- Arizona State University (US)
- Venue
- Zenodo
- Publisher
- Zenodo
- Published
- 2012-10-01
- Volume
- 4
- Issue
- 1
- Pages
- 11-48
- ISSN
- 2157-2100 (ISSN-L)
- OpenAlex
- W2142733603
- Cited by
- 139
- References
- 2
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 18 · 2023: 10 · 2022: 17 · 2021: 7 · 2020: 8 · 2019: 13 · 2018: 6 · 2017: 8 · 2016: 11 · 2015: 13 · 2014: 10 · 2013: 7 · 2012: 2
- FWCI
- 16.47
- Type
- journal article
- Language
- en
- Open access
- Locations
- Journal of Educational Data Mining (publishedVersion)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, other-oa)
- Zenodo (CERN European Organization for Nuclear Research)
- Educational Data Mining
- Funders
- U.S. Department of Education
- Princeton University
- University of Notre Dame
- University of Cambridge
- Institute of Education Sciences (R305C080015)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Education › Educational Assessment and Pedagogy
- Keywords
- evidence-centered design, educational assessment, educational data mining, psychometrics, assessment design, blended learning, online assessment, Cisco Networking Academy, information technology education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
