The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
A Systems Approach to Educational Testing
Abstract
Our concern in this paper is with the validity of educational tests when they are employed as critical measures of educational outcomes within a dynamic system. The problem of validity arises if an educational system adapts itself to the characteristics of the outcome measures. We introduce the concept of systemically valid tests as ones that induce curricular and instructional changes in education systems (and learning strategy changes in students) that foster the development of the cognitive traits that the tests are designed to measure. We analyze some general characteristics that contribute to or detract from a testing system's systemic validity, such as the use of direct rather than indirect assessment. We then apply these characteristics in developing a set of design principles for creating testing systems that are systemically valid. Finally, we provide an illustration of the proposed principles, by applying them to the design of a student assessment system. This design example addresses not only specifications for the tests, but also the means of teaching the process of assessment to users of the system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Situated Cognition and the Culture of Learning
Brown, J. S., Collins, A., & Duguid, P. (1989) · Educational Researcher · doi:10.3102/0013189x018001032 · Cited by 13,645
Self-citationAnderson, J. R., Boyle, C. F., & Reiser, B. J. (1985) · Science · doi:10.1126/science.228.4698.456 · Cited by 3,119
Gunderman, R. B. (2011) · Achieving Excellence in Medical Education · doi:10.1007/978-0-85729-307-7_10 · Cited by 2,892
Educational measurement, 3rd ed.
Linn, R. L. (1989) · Cited by 2,882
Moses, L. E., & Lindquist, E. F. (1951) · Journal of the American Statistical Association · doi:10.2307/2280411 · Cited by 1,661
Students’ preconceptions in introductory mechanics
Clement, J. (1982) · American Journal of Physics · doi:10.1119/1.12989 · Cited by 1,230
Bouwman, M. J. (1983) · European Journal of Operational Research · doi:10.1016/0377-2217(83)90060-7 · Cited by 1,064
Cognitive Processes in Writing
Gregg, L. W. (2016) · Routledge eBooks · doi:10.4324/9781315630274 · Cited by 931
A True Test: Toward More Authentic and Equitable Assessment
Wiggins, G. (2011) · Phi Delta Kappan · doi:10.1177/003172171109200721 · Cited by 820
Informal Reasoning and Education
(2012) · Routledge eBooks · doi:10.4324/9780203052228 · Cited by 702
Cognitive Processes in Writing
Clifford, J., Gregg, L. W., & Steinberg, E. R. (1983) · College Composition and Communication · doi:10.2307/358120 · Cited by 638
Research on conceptual understanding in mechanics
McDermott, L. C. (1984) · Physics Today · doi:10.1063/1.2916318 · Cited by 623
Curvilinear Motion in the Absence of External Forces: Naïve Beliefs About the Motion of Objects
McCloskey, M., Caramazza, A., & Green, B. (1980) · Science · doi:10.1126/science.210.4474.1139 · Cited by 586
The real test bias: Influences of testing on teaching and learning.
Frederiksen, N. (1984) · American Psychologist · doi:10.1037//0003-066x.39.3.193 · Cited by 531
Unlearning Aristotelian physics: a study of knowledge-based learning
DISESSA, A. (1982) · Cognitive Science · doi:10.1016/s0364-0213(82)80005-0 · Cited by 496
Diagnostic Monitoring of Skill and Knowledge Acquisition
(2013) · Routledge eBooks · doi:10.4324/9780203056899 · Cited by 468
Learning Issues for Intelligent Tutoring Systems
(1988) · Springer eBooks · doi:10.1007/978-1-4684-6350-7 · Cited by 370
The Computer as a Tool for Learning Through Reflection
Collins, A., & Brown, J. S. (1988) · Learning Issues for Intelligent Tutoring Systems · doi:10.1007/978-1-4684-6350-7_1 · Cited by 291
Self-citationDesigning Computer Games to Help Physics Students Understand Newton's Laws of Motion
White, B. Y. (1984) · Cognition and Instruction · doi:10.1207/s1532690xci0101_4 · Cited by 242
Beyond Standardization: State Standards and School Improvement
Darling‐Hammond, L., & Wise, A. E. (1985) · The Elementary School Journal · doi:10.1086/461408 · Cited by 233
A framework for a cognitive theory of writing
Collins, A., & Gentner, D. (1980) · Cited by 140
Self-citationSources of Difficulty in Understanding Newtonian Dynamics*
White, B. Y. (1983) · Cognitive Science · doi:10.1207/s15516709cog0701_2 · Cited by 115
Frederiksen, N. (1981) · ETS Research Report Series · doi:10.1002/j.2333-8504.1981.tb01267.x · Cited by 68
Raising the Stakes in Statewide Mandatory Minimum Competency Testing.
Corbett, H. D., & Wilson, B. M. (1989) · Cited by 19
Cited by
Enhancing teaching through constructive alignment
Biggs, J. (1996) · Higher Education · doi:10.1007/bf00138871 · Cited by 4,084
Validating the interpretations and uses of test scores
Kane, M. T. (2013) · Journal of Educational Measurement · doi:10.1111/jedm.12000 · Cited by 2,110
The Role of Assessment in a Learning Culture
Shepard, L. A. (2000) · Educational Researcher · doi:10.3102/0013189x029007004 · Cited by 2,161
Design Research: Theoretical and Methodological Issues
Collins, A., Joseph, D., & Bielaczyc, K. (2004) · Journal of the Learning Sciences · doi:10.1207/s15327809jls1301_2 · Cited by 1,923
Self-citationThe use of scoring rubrics: Reliability, validity and educational consequences
Jonsson, A., & Svingby, G. (2007) · Educational Research Review · doi:10.1016/j.edurev.2007.05.002 · Cited by 1,509
Metadata
- Authors
- Frederiksen, J. R., & Collins, A.
- Venue
- Educational Researcher
- Published
- 1989-12
- Volume
- 18
- Issue
- 9
- Pages
- 27
- ISSN
- 0013-189X (ISSN-L)
- OpenAlex
- W2051013971
- Cited by
- 648
- References
- 24
- FWCI
- 148.40
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.