The DOI matches its Crossref record; 42 of the 43 references it cites were found in a registry or an index.
Bias in grading: A meta-analysis of experimental research findings
Abstract
This article provides a meta-analysis of experimental research findings on the existence of bias in subjective grading of student work such as essay writing. Twenty-three analyses, from 20 studies, with a total of 1935 graders, met the inclusion criteria for the meta-analysis. All studies involved graders being exposed to a specific type of information about a student other than the student’s performance on a task. The hypothesized biasing characteristics included different race/ethnic backgrounds, education-related deficiencies, physical unattractiveness and poor quality of prior performance. The statistically significant overall between-groups effect size was g = 0.36. Moderator analyses showed no significant difference in effect size related to whether the work graded was from a primary school student or a university student. No one type of biasing characteristic showed a significantly higher effect size than other types. The results suggest that bias can occur in subjective grading when graders are aware of irrelevant information about the students.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Finally, assessment is indivisible from individual value judgements. No matter the structures and processes put in place, assessments are designed and evaluated by humans, with all their complex socio-cultural backgrounds, educational experiences, and intellectual and personal values (Malouff and Thorsteinsson 2016; Ryan and Viete 2009; Leathwood 2005; Stowell 2004).”
From A typology for a social justice approach to assessment: learning from universal design and culturally sustaining pedagogy (2019) · cites this paper as Malouff and Thorsteinsson 2016
In the citing paper
“Indeed, this was found to be a significant factor in student attainment outcomes when identity markers differed from those of academic staff (Malouff and Thorsteinsson 2016).”
From A typology for a social justice approach to assessment: learning from universal design and culturally sustaining pedagogy (2019) · cites this paper as Malouff and Thorsteinsson 2016
Citations
Cites
(2011) · Cited by 14,317
(2000) · Cited by 9,198
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Implicit social cognition: Attitudes, self-esteem, and stereotypes
Greenwald, A. G., & Banaji, M. R. (1995) · Psychological Review · doi:10.1037/0033-295x.102.1.4 · Cited by 6,682
The use of scoring rubrics: Reliability, validity and educational consequences
Jonsson, A., & Svingby, G. (2007) · Educational Research Review · doi:10.1016/j.edurev.2007.05.002 · Cited by 1,509
A hypothesis-confirming bias in labeling effects
Darley, J. M., & Gross, P. H. (1983) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.44.1.20 · Cited by 1,173
Cooper, W. H. (1981) · Psychological Bulletin · doi:10.1037/0033-2909.90.2.218 · Cited by 492
Beauty is talent: Task evaluation as a function of the performer's physical attractiveness.
Landy, D., & Sigall, H. (1974) · Journal of Personality and Social Psychology · doi:10.1037/h0036018 · Cited by 449
Eurasian Journal of Educational Research
(2015) · Eurasian Journal of Educational Research · doi:10.14689/ejer.2015 · Cited by 299
Name stereotypes and teachers' expectations.
Harari, H., & McDavid, J. W. (1973) · Journal of Educational Psychology · doi:10.1037/h0034978 · Cited by 214
Research works printed but not linked: 7
- 10.3102/00028312022002175 · DOI
- Belsey, C. (1988). Marking by numbers. AUT women , No. 15 (pp. 1–2). London, UK: Association of University Teachers.
- Borenstein M., 2005, Comprehensive Meta-Analysis Version 2
- *Gerritson, M. (2013). Rubrics as a mitigating instrument for bias in the grading of student writing (Doctoral dissertation, Walden University).
- Haeran J., 2010, College Teaching, 57, 51
- *King J. L., 1998, Educational Research Quarterly, 22, 13
- Warren Piper D., 1996, Examination practices and procedures in Australian universities
Cited by
Sorted by citations
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777
The need to disentangle assessment and feedback in higher education
Winstone, N., & Boud, D. J. (2020) · Studies in Higher Education · doi:10.1080/03075079.2020.1779687 · Cited by 369
Novice Programmers and Introductory Programming
Robins, A. V. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108654555.013 · Cited by 191
The Cambridge Handbook of Computing Education Research
Fincher, S., Robins, A. V., Margulieux, L. E., Morrison, B. B., Shephard, K., Loui, M., Borrego, M., Lancaster, T., Falkner, K., Sheard, J., et al. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108654555 · Cited by 141
Experimental Evidence on Teachers’ Racial Bias in Student Evaluation: The Role of Grading Scales
Quinn, D. M. (2020) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373720932188 · Cited by 124
Metadata
- Authors
- Malouff, J. M., & Thorsteinsson, E. B.
- Affiliations
- University of New England (AU)
- Corresponding author
- Malouff, John M
- Venue
- Australian Journal of Education
- Publisher
- SAGE Publications
- Published
- 2016-09-21
- Volume
- 60
- Issue
- 3
- Pages
- 245-256
- ISSN
- 0004-9441 (ISSN-L)
- 2050-5884
- OpenAlex
- W2512722101
- Cited by
- 222
- References
- 43
- Citations by year
- 2026: 22 · 2025: 29 · 2024: 44 · 2023: 21 · 2022: 26 · 2021: 15 · 2020: 13 · 2019: 42 · 2018: 6 · 2017: 3
- FWCI
- 6.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Australian Journal of Education (publishedVersion)
- EPubDev Repository (submittedVersion)
- RUNE (Research UNE) (submittedVersion)
- Topics
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
- Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
- Keywords
- grading bias, subjective grading, essay writing, student assessment, race and ethnicity, physical attractiveness, prior performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
