CoChat
Sign Up
Cited by 33Educational Assessment and PedagogyPsychometric Methodologies and TestingStudent Assessment and Feedback

The DOI matches its Crossref record.

Who should mark what? A study of factors affecting marking accuracy in a biology examination

Suto, Irenka, Nádas, Rita, Bell, John·2011·Research Papers in Education
DOI 10.1080/02671520902721837
Open this paper in CoChat — free account

Abstract

Accurate marking is crucial to the reliability and validity of public examinations, in England and internationally. Factors contributing to accuracy have been conceptualised as affecting either marking task demands or markers’ personal expertise. The aim of this empirical study was to develop this conceptualisation through investigating the relative effects on accuracy of some key contributory factors. This was done in the context of an international biology examination designed for 16‐year‐olds. Forty‐two markers participated, comprising five groups: (1) experienced examiners, (2) biology teachers, (3) graduates in biology, (4) graduates in other subjects, and (5) non‐graduates. The design enabled the relative effects on accuracy of the following factors to be elicited: marking experience, teaching experience, highest education in a relevant subject, highest education in any subject and gender. The significance of markers’ accuracy on standardisation samples was also investigated. Twenty‐three examination questions were explored, varying in format, number of marks, difficulty for candidates and cognitive marking strategy complexity. All markers marked identical response samples for each question. The data were modelled with logistic regression and AN(C)OVA, generating a refined model of marking accuracy. This may prove useful for evidence‐based decisions surrounding marker selection and training.

Overview

Integrity signals

Registry

Crossref · 10.1080/02671520902721837

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 42

  1. 10.1080/0969594042000304627 · DOI
  2. Bell, J., Bramley, T., Claessen, M. and Raikes, N. Quality control of marking: Some models and simulations. Paper presented at the annual conference of the International Association for Educational Assessment. May21–26. Singapore
  3. Bramley T., 2007, Research Matters: A Cambridge Assessment Publication, 4, 22
  4. Bramley, T. Mark scheme features associated with different levels of marker agreement. Paper presented at the annual conference of the British Educational Research Association (BERA). September. Edinburgh: Heriot‐Watt University.
  5. 10.1080/0013191810330105 · DOI
  6. 10.1080/0013191800320207 · DOI
  7. Broadfoot P.M., 1996, Education, assessment and society
  8. Clark L., 2008, Mail Online
  9. Eckstein M.A., 1993, A comparative study of secondary school examinations (Research working paper), 10.2307/j.ctt211qvfm · DOI
  10. Frean A., 2008, Times Online
  11. Gilovich T., 2002, Heuristics and biases: The psychology of intuitive judgement, 10.1017/CBO9780511808098 · DOI
  12. Greatorex, J. Do examiners’ approaches to marking change between when they first begin marking and when they have marked many scripts? Paper presented at the annual conference of the British Educational Research Association. September6–9. UK: University of Warwick.
  13. Greatorex J., 2004, Research in Education, 71, 25, 10.7227/RIE.71.4 · DOI
  14. Greatorex, J., Nádas, R., Suto, W.M.I. and Bell, J. Exploring how the cognitive strategies used to mark examination questions relate to the efficacy of examiner training. Paper presented at the European Conference on Educational Research. September16–22, Ghent, Belgium.
  15. Greatorex, J. and Suto, W.M.I. An empirical exploration of human judgement in the marking of school examinations. Paper presented at the annual conference of the International Association for Educational Assessment. May21–26. Singapore
  16. Greatorex, J., Suto, W.M.I. and Nadas, R. Exploring the role of human judgement in examination marking: findings from some empirical studies. Paper presented at the annual conference of the Association for Language Testing. April, Cambridge. Europe.
  17. 10.1207/s15434311laq0202_2 · DOI
  18. Kounine L., 2008, Report for Reform Think‐tank
  19. Laming D., 2004, Human judgement: The eye of the beholder
  20. 10.1080/03054980802649360 · DOI
  21. Meadows, M. Can we predict who will be a reliable marker? Paper presented at the annual conference of the International Association for Educational Assessment. May21–26. Singapore
  22. Meadows, M. and Billington, L. The right attitude for marking? Paper presented at the annual conference of the British Educational Research Association. September5–8. London
  23. Meadows, M. and Wheadon, C. Selecting the conscientious marker – A study of marking reliability in GCSE English. Paper presented at the annual conference of the International Association for Educational Assessment. September16–21, Baku. Azerbaijan
  24. 10.1111/j.2044-8279.1978.tb02385.x · DOI
  25. 10.1111/j.2044-8279.1982.tb02503.x · DOI
  26. 10.1080/0141192960220403 · DOI
  27. Paton G., 2008, Telegraph Online
  28. Pinot de Moira A., 2002, Research in Education, 67, 79, 10.7227/RIE.67.8 · DOI
  29. Powers D., 1998, Qualifying essay readers for an on‐line scoring network (ETS RR‐98‐20)
  30. 2008, GCSE, GCE and AEA code of practice (April 2008)
  31. Raikes N., 2007, Research Matters: A Cambridge Assessment Publication, 4, 34
  32. Royal‐Dawson L., 2005, Is teaching experience a necessary condition for markers of key Stage 3 English?
  33. 10.2307/329895 · DOI
  34. Stahl J.A., 1996, Objective measurement. Theory into practice, 113
  35. Suto W.M.I., 2008, Research Matters: A Cambridge Assessment Publication, 5, 6
  36. 10.1080/01411920701492050 · DOI
  37. 10.1080/09695940701876177 · DOI
  38. Suto W.M.I., 2007, Research Matters: A Cambridge Assessment Publication, 4, 2
  39. 10.1080/02671520701755499 · DOI
  40. Valentine C.W., 1932, The reliability of examinations: An enquiry with special reference to the entrance examinations to secondary schools, the school certificate examinations, and the award of scholarships at universities
  41. 10.1177/026553229301000306 · DOI
  42. Wood R., 1991, Assessment and testing

Cited by

See all 33

Metadata

Authors
Suto, I., Nádas, R., & Bell, J.
Corresponding author
Suto, Irenka
ORCID
Suto, Irenka
Venue
Research Papers in Education
Publisher
Informa UK Limited
Published
2011-03
Volume
26
Issue
1
Pages
21-51
ISSN
  • 0267-1522 (ISSN-L)
  • 1470-1146
DOI
10.1080/02671520902721837
OpenAlex
W2056361398
Cited by
33
References
0
Citations by year
2026: 3 · 2025: 1 · 2024: 1 · 2022: 3 · 2021: 1 · 2020: 2 · 2019: 3 · 2017: 3 · 2016: 1 · 2014: 3 · 2013: 3 · 2012: 2
FWCI
12.75
Type
journal article
Language
en
Open access
closed
Locations
Research Papers in Education (publishedVersion)
Topics
  • Social Sciences › Education › Educational Assessment and Pedagogy
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
  • Social Sciences › Education › Student Assessment and Feedback
Keywords
examiners, biology examinations, marker selection, teaching experience, subject matter expertise, gender differences, international examinations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Who should mark what? A study of factors affecting marking accuracy in a biology examination. https://app.cochat.ai/papers/10.1080/02671520902721837
The paper
Suto, I., Nádas, R., & Bell, J. (2011). Who should mark what? A study of factors affecting marking accuracy in a biology examination. Research Papers in Education. https://doi.org/10.1080/02671520902721837