Indeterminacy in the use of preset criteria for assessment and grading
Open accessAbstract
When assessment tasks are set for students in universities and colleges, a common practice is to advise them of the criteria that will be used for grading their responses. Various schemes for using multiple criteria have been widely advocated in the literature. Each scheme is designed to offer clear benefits for students. Breaking down holistic judgments into more manageable parts is seen as a way to increase openness for students and achieve more objectivity in grading. However, such approaches do not adequately represent the full complexity of multi‐criterion qualitative judgments, and can lead to distorted grading decisions. Six anomalies in the ways assessors approach the grading task are identified, together with several likely contributing factors. Overall, the conclusion is that explicit grading models do not have as strong a theoretical foundation as is commonly supposed, and that holistic appraisal merits further investigation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Motivational processes affecting learning.
Dweck, C. S. (1986) · American Psychologist · doi:10.1037/0003-066x.41.10.1040 · Cited by 6,971 · see citing works
Formative assessment and the design of instructional systems
Sadler, D. R. (1989) · Instructional Science · doi:10.1007/bf00117714 · Cited by 4,285 · see citing works
Self-citationBUTLER, R. (1988) · British Journal of Educational Psychology · doi:10.1111/j.2044-8279.1988.tb00874.x · Cited by 506 · cited 1× · see citing works
“For example, there may be differences that reflect the extent to which students use their grades as a means of establishing their performance rankings among peers (Dweck, 1986; Butler, 1987, 1988 ), or differences that are specific to discipline (Hodgen & Marshall, 2005).”
Transforming holistic assessment and grading into a vehicle for complex learning
Sadler, D. R. (2009) · Assessment, Learning and Judgement in Higher Education · doi:10.1007/978-1-4020-8905-3_4 · Cited by 162 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 15
- Burt, C. (1920) Tests of educational attainments. Memorandum 3 in C. Burt, (1947). Mental and scholastic tests (3rd ed.) (London, Staples), 285-366.
- Butler, R. (1987) Task-involving and ego-involving properties of evaluation: effects of different feedback conditions on motivational perceptions, interest and performance, Journal of Educational Psychology, 79, 474-482.
- Cast, B. M. D. (1939) The efficiency of different methods of marking English composition: Part 1, British Journal of Educational Psychology, 9, 257-269.
- Coombs, C. H. (1964) A theory of data (New York, John Wiley).
- Elbow, P. (1973) Writing without teachers (New York, Oxford University Press).
- Ericsson, K. A. & Smith, J. (Eds) (1991) Toward a general theory of expertise: prospects and limits (New York, Cambridge University Press).
- Hodgen, J. & Marshall, B. (2005) Assessment for learning in English and mathematics: a comparison, Curriculum Journal, 16, 153-176.
- Marshall, M. S. (1968) Teaching without grades (Corvallis, Oregon, Oregon State University Press).
- Meehl, P. E. (1996) Clinical versus statistical prediction: a theoretical analysis and a review of the evidence (New Preface) (Lanham, Maryland, Rowan & Littlefield/Jason Aronson). (Original work published 1954).
- Morgan, C., Dunn, L., Parry S. & O'Reilly, M. (2004) The student assessment handbook: new directions in traditional and online assessment (London, RoutledgeFalmer).
- Polanyi, M. (1962) Personal knowledge (London, Routledge & Kegan Paul).
- Sadler, D. R. (1980) Conveying the findings of evaluative inquiry, Educational Evaluation and Policy Analysis, 2(2), 53-57.
- Sadler, D. R. (1983) Evaluation and the improvement of academic learning, Journal of Higher Education, 54, 60-79.
- Sadler, D. R. (1987) Specifying and promulgating achievement standards, Oxford Review of Education, 13, 191-209.
- Walvoord, B. E. & Anderson, V. J. (1998) Effective grading: a tool for learning and assessment. (Etobicoke, Ontario, John Wiley).
Cited by
Sorted by citations
Beyond feedback: developing student capability in complex appraisal
Sadler, D. R. (2010) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930903541015 · Cited by 1,138 · see citing works
Self-citationA review of feedback models and typologies: Towards an integrative model of feedback elements
Panadero, E., & Lipnevich, A. A. (2022) · Educational Research Review · doi:10.1016/j.edurev.2021.100416 · Cited by 389 · see citing works
Does student engagement in self-assessment calibrate their judgement over time?
Boud, D., Lawson, R., & Thompson, D. G. (2013) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2013.769198 · Cited by 221 · see citing works
A critical review of the arguments against the use of rubrics
Panadero, E., & Jonsson, A. (2020) · Educational Research Review · doi:10.1016/j.edurev.2020.100329 · Cited by 191 · see citing works
Bloxham, S., den-Outer, B., Hudson, J., & Price, M. (2016) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2015.1024607 · Cited by 135 · see citing works
Using learning analytics to understand student perceptions of peer feedback
Misiejuk, K., Wasson, B., & Egelandsdal, K. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106658 · Cited by 98 · see citing works
Walton, J., Bearman, M., Crawford, N., Tai, J., & Boud, D. (2026) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2025.2570328 · Cited by 28 · see citing works
“These trajectories of judgement are not simply cognitive events, but involve tacit evaluation (Sadler 2009), mediated by material constraints and affordances (Esterhazy and Damşa 2019).”
Designing AI-resilient assessment in higher education: a four-pillar conceptual framework
Nikolić, D., & Basta Nikolić, M. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1841682 · Cited by 1 · see citing works
“These limitations suggest that the primary response should shift from post hoc detection toward assessment designs that require students to demonstrate reasoning and application (Sadler, 2009).”
- Authors
- Sadler, D. R.
- Affiliations
- Griffith University (AU)
- Corresponding author
- Sadler, D. Royce
- ORCID
- Sadler, D. Royce
- Venue
- Assessment & Evaluation in Higher Education
- Publisher
- Informa UK Limited
- Published
- 2009-03-18
- Volume
- 34
- Issue
- 2
- Pages
- 159-179
- ISSN
- 0260-2938 (ISSN-L)
- 1469-297X
- OpenAlex
- W2102274116
- Cited by
- 489
- References
- 4
- Citations by year
- 2026: 34 · 2025: 24 · 2024: 27 · 2023: 37 · 2022: 24 · 2021: 30 · 2020: 53 · 2019: 22 · 2018: 21 · 2017: 20 · 2016: 23 · 2015: 35 · 2014: 36 · 2013: 31 · 2012: 27
- FWCI
- 65.15
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- Assessment & Evaluation in Higher Education (publishedVersion)
- Griffith Research Online (submittedVersion)
- Topics
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Education › Educational Assessment and Pedagogy
- Social Sciences › Education › Evaluation of Teaching Practices
- Keywords
- grading, evaluation criteria, multiple criteria, holistic assessment, qualitative judgment, higher education assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
