Let’s stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Unreliability in marking is well documented, yet we lack studies that have investigated assessors’ detailed use of assessment criteria. This project used a form of Kelly’s repertory grid method to examine the characteristics that 24 experienced UK assessors notice in distinguishing between students’ performance in four contrasting subject disciplines: that is their implicit assessment criteria. Variation in the choice, ranking and scoring of criteria was evident. Inspection of the individual construct scores in a sub-sample of academic historians revealed five factors in the use of criteria that contribute to marking inconsistency. The results imply that, whilst more effective and social marking processes that encourage sharing of standards in institutions and disciplinary communities may help align standards, assessment decisions at this level are so complex, intuitive and tacit that variability is inevitable. We conclude that universities should be more honest with themselves and with students, and actively help students to understand that application of assessment criteria is a complex judgement and there is rarely an incontestable interpretation of their meaning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, even experts vary in their interpretation of standards and use of marking criteria, especially when they are not made explicit from the outset (Bloxham et al. 2016).”
From Developing evaluative judgement: enabling students to make decisions about the quality of work (2018) · cites this paper as Bloxham et al. 2016
Citations
Cites
Formative assessment and the design of instructional systems
Sadler, D. R. (1989) · Instructional Science · doi:10.1007/bf00117714 · Cited by 4,285 · cited 2×
“Similar notions have been theorised such as ‘teachers conceptions of quality’ (Sadler 1989, 127), assessors’ ‘evaluative frameworks’ (Broad 2003), ‘assessors’ interpretive frameworks’ (Shay 2005, 665), ‘pre-formed knowledge structures’ (Crisp 2008, 250) and ‘ways of understanding the world’ (Read,…”
BackgroundA manual for repertory grid technique
Fransella, F., Bell, R., & Bannister, D. (1977) · Thammasat University Digital Collections · Cited by 1,574
Indeterminacy in the use of preset criteria for assessment and grading
Sadler, D. R. (2009) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930801956059 · Cited by 489 · cited 6×
“This tends to reinforce arguments about the holistic nature of judgement (Sadler 2009), where assessors will balance different aspects of a piece of work.”
BackgroundInfluentialWolf, A. (2001) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 389
Assessment in Higher Education
Mathews, J. C., & Heywood, J. (1978) · British Journal of Educational Studies · doi:10.2307/3120746 · Cited by 299
What We Really Value: Beyond Rubrics in Teaching and Assessing Writing
Broad, B. (2003) · Digital Commons - USU (Utah State University) · Cited by 264
Assessing the Undergraduate Dissertation
Webster, F., Pepper, D., & Jenkins, A. (2000) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930050025042 · Cited by 180 · cited 2×
“…Education, 2016 Vol. 41, No. 3, 466–481, http://dx.doi.org/10.1080/02602938.2015.1024607 Greatorex 2000; Price 2005; Read, Francis, and Robson 2005; Webster, Pepper, and Jenkins 2000), including the use of implicit assessment criteria not shared explicitly with students or other assessors…”
BackgroundInfluentialAssessment standards: the role of communities of practice and the scholarship of assessment
Price *, M. (2005) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930500063793 · Cited by 167 · cited 5×
“In addition, assessors use personal criteria beyond or different to those stated (Baume, Yorke and Coffey 2004; Broad 2003; Greatorex 2000; Price 2005; Read, Francis and Robson 2005; Webster, Pepper and Jenkins 2000) including the use of implicit assessment criteria not shared explicitly with students or other assesso…”
BackgroundInfluentialSelf-citationMark my words: the role of assessment criteria in UK higher education grading practices
Bloxham, S., Boyd, P., & Orr, S. (2011) · Studies in Higher Education · doi:10.1080/03075071003777716 · Cited by 141 · cited 3×
“This study further clarifies the notion of personalised ‘standards frameworks’ (Bloxham, Boyd, and Orr 2011), which are conceived of as the lens through which individuals read student work.”
BackgroundSelf-citationSummative assessment: dealing with the ‘measurement fallacy’
Yorke, M. (2010) · Studies in Higher Education · doi:10.1080/03075070903545082 · Cited by 122 · cited 1×
“Short of turning our assessment methods into standardised tests, we have to live with a large element of unreliability and a recognition that grading is judgement and not measurement (Yorke 2011).”
Method
Research works printed but not linked: 1
- Greatorex J. 2000. “Is the Glass Half Full or Half Empty? What Examiners Really Think of Candidates’ Achievement.” Paper presented at the British Educational Research Association Conference Cardiff September 7–10 2000.
Cited by
Developing evaluative judgement: enabling students to make decisions about the quality of work
Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018) · Higher Education · doi:10.1007/s10734-017-0220-3 · Cited by 810
“However, even experts vary in their interpretation of standards and use of marking criteria, especially when they are not made explicit from the outset (Bloxham et al. 2016).”
BackgroundAssessment for inclusion: rethinking contemporary strategies in assessment design
Tai, J., Ajjawi, R., Bearman, M., Boud, D., Dawson, P., & Jorre de St Jorre, T. (2023) · Higher Education Research & Development · doi:10.1080/07294360.2022.2057451 · Cited by 133
Does the use of summative peer assessment in collaborative group work inhibit good judgement?
Sridharan, B., Tai, J., & Boud, D. (2019) · Higher Education · doi:10.1007/s10734-018-0305-7 · Cited by 106
“However, it is well documented that reliability and validity of marking is a big problem even amongst experienced assessors (Bloxham et al. 2016).”
BackgroundInfluentialCan a rubric do more than be transparent? Invitation as a new metaphor for assessment criteria
Bearman, M., & Ajjawi, R. (2021) · Studies in Higher Education · doi:10.1080/03075079.2019.1637842 · Cited by 83
Good and bad practices in rubrics: the perspectives of students and educators
Chan, C. Y. Z., & Ho, S. S. M. (2019) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2018.1522528 · Cited by 83
Metadata
- Authors
- Bloxham, S., den-Outer, B., Hudson, J., & Price, M.
- Affiliations
- University of Cumbria (GB)
- ,
- Oxford Brookes University (GB)
- Corresponding author
- Bloxham, Sue
- ORCID
- Price, Margaret
- Venue
- Assessment & Evaluation in Higher Education
- Publisher
- Informa UK Limited
- Published
- 2015-03-23
- Volume
- 41
- Issue
- 3
- Pages
- 466-481
- ISSN
- 0260-2938 (ISSN-L)
- 1469-297X
- OpenAlex
- W1983517965
- Cited by
- 135
- References
- 32
- Citations by year
- 2026: 16 · 2025: 12 · 2024: 17 · 2023: 11 · 2022: 10 · 2021: 14 · 2020: 8 · 2019: 12 · 2018: 20 · 2017: 10 · 2016: 2 · 2015: 3
- FWCI
- 58.31
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Assessment & Evaluation in Higher Education (publishedVersion)
- Insight (University of Cumbria) (acceptedVersion)
- Radar (Oxford Brookes University) (submittedVersion)
- Topics
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Education › Educational Assessment and Pedagogy
- Social Sciences › Education › Evaluation of Teaching Practices
- Keywords
- evaluation criteria, academic historians, repertory grid technique, tacit knowledge, higher education assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Let’s stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria. https://app.cochat.ai/papers/10.1080/02602938.2015.1024607
- The paper
- Bloxham, S., den-Outer, B., Hudson, J., & Price, M. (2016). Let’s stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2015.1024607
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.