'I know a 2:1 when I see it': Understanding criteria for degree classifications in franchised university programmes

“…and teachers develop the equivalent of such scripts where knowledge about what makes for quality becomes linked through repeated exposure to work, consideration of standards (whether these are implicit or explicit) and the need to justify decisions (as assessors do post hoc (Ecclestone 2001)).”

“There was, and is, pressure for institutions to maintain high and consistent academic standards within the context of a massive expansion of higher education (Lucas & Webster, 1998), amplified by increasing student hunger for and expectations of high grades (Ecclestone, 2001).”

“Their judgements become more intuitive than conscious, as they develop ‘mental models’ of marking which they apply regardless of marking guidance (Ecclestone 2001).”

“The notion of students developing increasingly complex levels of understanding and knowledge application is already well accepted in the design of assessment criteria, where standards to be met within a programme across all year levels identify an identical or similar action, but will be worded to reflect the level of…”

“From this perspective, written assessment criteria have limited power to secure national standards, as their interpretation will be determined locally (Knight 2006 ) by tutors drawing on their experience and, therefore, their differing tacit knowledge of disciplinary standards (Ecclestone 2001 ; Knight and Yorke 2003…”

“In a small-scale empirical study, Ecclestone (2001) found that novices were more rule-governed than competent or expert assessors, though the relationship between the level of competence of the assessor and the actual assessment outcomes was not clear-cut.”

“At the same time it is important to recognize that in practice, judgements are filtered through socialization into academic communities (Wolf, 1995; Ecclestone, 2001; Elton & Johnston, 2002).”

“Ecclestone (2001) considered assessment boards in Higher Education to be communities of academics and cites tacit knowledge as a feature of these communities.”

“Other problems identified in the literature are that markers may ignore the set criteria (Ecclestone 2001; Price and Rust 1999; Smith and Coombe 2006); they may differ in their interpretations of criteria or they may have higher standards than those set by the institution (Price 2005).”

“Sadler (2005) and Ecclestone (2001) similarly argue that academics continue to employ tacit judgements regardless of induction into the assessment process by conventional means.”

“Other problems with assessment grading in higher education contexts identified in previous research include markers developing fixed marking habits over time, and markers ignoring the assessment criteria (Ecclestone 2001; Smith and Coombe 2006).”

“Ecclestone (2001) is another proponent of the view that the judgments which underpin assessment are qualitatively different. Her study entailed some important differences from that undertaken by Suto and Greatorex (2008) yet her conclusions bear some striking similarities.”

“It is likely that when students know exactly what to do, their motivation, learning and performance will increase significantly (see for example Ecclestone 2001).”

“From this perspective, written assessment criteria have limited power to secure national standards, as their interpretation will be determined locally by tutors drawing on their experience and, therefore, their differing tacit knowledge of disciplinary standards (Price and Rust 1999; Ecclestone 2001; Knight and Yorke…”

“In addition, assessors tend to rely on heuristics and ignore (or decide to not adopt) written criteria in rubrics (Price and Rust 1999; Ecclestone 2001; Smith and Coombe 2006; Sadler 2009).”

“Ecclestone (2001) concurred that this tacit knowledge contributed to experts demonstrating scepticism towards the collaborative moderation process as a way of securing reliable judgements.”

“Inter-rater reliability may be influenced by teaching experience (Weigle 1999; Ecclestone 2001; Price 2005) and training (Weigle 1999).”

“The external examiners distanced themselves from uncalibrated internalised standards, concurring, it seems, that ‘there is a fine balance between a genuine ability to recognize quality of work apparently intuitively, and erratic interpretation’ (Ecclestone 2001, 305).”

“It may be that, in consequence, these experienced graders may become more intuitive than conscious when grading, thereby not relying and adhering to a given grading rubric as closely as an instructor with less grading experience (Ecclestone, 2001).”

“In line with this contention, Ecclestone (2001) observed staff to ‘compensate’ for the assessment criteria in order to take into account their perception of the student’s application, conscientiousness, personal pressures, personal progress and contributions during tutorials.”

“They have the potential to motivate and improve learning along with improving the standard of work students produce (Ecclestone, 2001).”

“…globally based on the quality of their teaching and learning outcomes (Organisation for Economic Co-Operation and Development, 2010), universities have prioritised the need for increased transparency in assessment practices (Ecclestone, 2001; Mansell, James & The Assessment Reform Group, 2009).”

“The dominant techno-rational transparency approach in higher education, driven by a push for accountability (Bearman and Ajjawi 2021; O’Donovan, Price, and Rust 2004) and the belief in explicit assessment representation (Ecclestone 2001), shapes current policies.”

“…certainly of technical stuff (JR) The fact that both lecturers appear to be ‘harder’ markers with students who pursue their area of practice appears to relate to the unconscious compensation tactics described by Ecclestone (2001) that can work for or (as in this case) against the student.”

“There have been persistent concerns over the validity, reliability and fairness of assessment in HE (see for example, Ecclestone 2001; Fleming 1999; Knight 2002a; Yorke 2008, 2011), and James (2003, 197) argues that ‘assessment is possibly one of the least sophisticated aspects of university…”

“Ecclestone (2001) argues that when students know what they are expected to do, their motivation, and standard of work will improve.”

“In contrast to Schraw and Moshman (1995), the models of Ecclestone (2001) and Suto and Greatorex (2008) regarding raters’ cognitive operations in marking processes coincided in identifying a shift from mental processes characteristic of explicit knowledge to others more representative of tacit…”

“This can also have positive effects on students’ motivation, learning and performance (Ecclestone 2001).”

“However, in one comparison between ‘novice’, ‘competent’ and ‘expert’ markers, the competent and expert markers had more changes made to the grades they awarded after double marking and moderation than did the novice markers (Ecclestone, 2001).”

“Ainsi, si les enseignants sont assez unanimes pour identifier les critères pertinents pour classer les étudiants (l’analyse critique, par exemple), on observe que des comportements précis très différents sont retenus comme indicateurs de ce critère selon la discipline enseignée et même parfois selon les théories privi…”

“…assessment literature researchers have made a similar case for assessment as a social and collaborative practice when it comes to complex, open-ended assessment tasks, to enhance the quality of assessment through discussion and sharing experiences (Ecclestone 2001; Moss, Schutz, and Collins 1998).”

“The adoption of explicit marking criteria, rubrics and taxonomies describing categories of achievement can all be adopted to counter these problems, but work has shown these methods can be undermined by marker’s fixed habits (Wolf 1995), by staff ignoring standards (Baume, Yorke, and Coffey 2004; Ecclestone 2001; Pric…”

“…(Wright, 1991), Cuff, 1991), (Vernez-Moudon, 1992), (Lang, 1994, p. 255), (Alexander, Neis, Anninou, & King, 1987), (Duggan & Mitchell, 1997), (Ecclestone, 2001), (Gallaudet University, 2002), (Campbell, 2002, p. 9), (Creswell, 2003), (The Office of Academic Planning & Assessment, 2005),…”

“Given limited resources, markers must often mark large numbers of essays while meeting specific deadlines and fulfilling other aspects of their profession (Bloxham, 2009; Ecclestone, 2001; Smith & Coombe, 2006).”

“My best intentions of allocating ‘research time’ have fallen foul of other competing demands and I’m afraid that other than ‘sorting’ of notes and articles into those that I may be using and those that I’ve decided that I won’t (including Ecclestone, 2001 ; Laugharne, 2002 ; and Rust, 2002 ), I haven’t done a great de…”

“However, ‘inherent quality’ can be a slippery concept, as indicated by the muchcoined saying ‘I know a 2:1 when I see it’ (Ecclestone, 2001).”

“(Wolf 1993, 21) Subsequent studies drew similar conclusions (Ecclestone 2001; Price 2005).”

“A growing body of literature around the experience of criterion-referenced assessment has elaborated on problematic as well as beneficial aspects (Greatorex, 1999; Price & Rust, 1999; Ecclestone, 2001; Rust, 2001; Dunn et al., 2002; Knight, 2002a, b; Hornby, 2003; Miller, 2003; Sadler, 2003, 2005; Taras, 2003; Orsmond…”

“The adoption of such techno-rationalist discourses may have been in an attempt to avoid the conflation of ‘connoisseurship’ with absolute values (Hardy, 2006), elitism, unaccountability and mystery (Ecclestone, 2001).”

“Ecclestone (2001) suggested that well-designed assessment criteria can motivate students to improve the standard of work they produce. nonetheless, researchers continue to debate the reliability and validity of these moderation practices.”

“A precise de fi nition of assessment criteria helps teachers draw more reliable judgement and make assessment more amendable to moderation, thereby enhancing the reliability of the assessment (Ecclestone 2001).”

“…of methodologically different studies, from judgement analysis to observations of meetings to interviews with staff, with the consequent use of other, nonpublished criteria (e.g., Baird, 2002; Crisp, 2007; Ecclestone, 2001; Greatorex, 2002; Yorke, Bridges, & Woolf, 2000; Webster et al., 2000).”

“An analogous division is also posited by Cooksey, Freebody and Wyatt-Smith (2007), Ecclestone (2001), Keren and Teigen (2004), Lumley (2005), and Shirazi (2012), among others.”

“Box (2004) and Eccleston (2001) suggest distributing samples of work to students showing different standards.”