Grading by AI makes me feel fairer? How different evaluators affect college students’ perception of fairness
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
Introduction: In the field of education, new technologies have enhanced the objectivity and scientificity of educational evaluation. However, concerns have been raised about the fairness of evaluators, such as artificial intelligence (AI) algorithms. This study aimed to assess college students’ perceptions of fairness in educational evaluation scenarios through three studies using experimental vignettes. Methods: Three studies were conducted involving 172 participants in Study 1, 149 in Study 2, and 145 in Study 3. Different evaluation contexts were used in each study to assess the influence of evaluators on students’ perception of fairness. Information transparency and explanations for evaluation outcomes were also examined as potential moderators. Results: Study 1 found that different evaluators could significantly influence the perception of fairness under three evaluation contexts. Students perceived AI algorithms as fairer evaluators than teachers. Study 2 revealed that information transparency was a mediator, indicating that students perceived higher fairness with AI algorithms due to increased transparency compared with teachers. Study 3 revealed that the explanation of evaluation outcomes moderated the effect of evaluator on students’ perception of fairness. Specifically, when provided with explanations for evaluation results, the effect of evaluator on students’ perception of fairness was lessened. Discussion: This study emphasizes the importance of information transparency and comprehensive explanations in the evaluation process, which is more crucial than solely focusing on the type of evaluators. It also draws attention to potential risks like algorithmic hegemony and advocates for ethical considerations, including privacy regulations, in integrating new technologies into educational evaluation systems. Overall, this study provides valuable theoretical insights and practical guidance for conducting fairer educational evaluations in the era of new technologies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“However, for most users, the complex internal operations of AI algorithm are not easy to understand so there is a “black box” problem for it as an evaluator (Adadi and Berrada, 2018; Guidotti et al., 2018; Rudin, 2019; Luo et al., 2021; Zednik and Boelsen, 2022).”
BackgroundFrom local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · cited 1×
“The interpretability of an AI algorithm refers to its ability to be understood and explained, and it is claimed that its operational mechanism, process, and result can be clearly understood by users (Lundberg et al., 2020; Shin, 2021; Cong et al., 2022; Sun, 2022).”
BackgroundPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1×
“As a result, compared to the human evaluation, which can be observed and understood more easily, the unobservable and unintelligible characteristics of AI algorithm evaluation were more likely to raise doubts about the fairness of the evaluation (Burrell, 2016).”
ResultCan you see the real me? A self-based model of authentic leader and follower development
Gardner, W. L., Avolio, B. J., Luthans, F., May, D. R., & Walumbwa, F. (2005) · The Leadership Quarterly · doi:10.1016/j.leaqua.2005.03.003 · Cited by 2,232
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“Some scholars believed that explanation was the logical and meaningful information provided by the decision maker about the result, which could explain the relationship between the input contents and output results (Goodman and Flaxman, 2017).”
BackgroundAlgorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Shin, D. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102551 · Cited by 1,548
Awad, N. F., & Krishnan, M. S. (2006) · MIS Quarterly · doi:10.2307/25148715 · Cited by 1,252
Cited by
Assessing AI adoption in developing country academia: A trust and privacy-augmented UTAUT framework
Rana, M. M., Siddiqee, M. S., Sakib, M. N., & Ahamed, M. R. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e37569 · Cited by 143
Cai, L., Msafiri, M. M., & Kangwa, D. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-13112-0 · Cited by 104
The Impact of AI on the Personal and Collaborative Learning Environments in Higher Education
Msambwa, M. M., Wen, Z., & Daniel, K. (2025) · European Journal of Education · doi:10.1111/ejed.12909 · Cited by 76
Daniel, K., Msambwa, M. M., & Wen, Z. (2025) · European Journal of Education · doi:10.1111/ejed.70036 · Cited by 71
Kangwa, D., Msafiri, M. M., & Fute, A. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70109 · Cited by 22
Metadata
- Authors
- Chai, F., Ma, J., Wang, Y., Zhu, J., & Han, T.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2024-02-02
- Volume
- 15
- Pages
- 1221177-1221177
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W4391481267
- Cited by
- 76
- References
- 24
- FWCI
- 29.08
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Grading by AI makes me feel fairer? How different evaluators affect college students’ perception of fairness. https://app.cochat.ai/papers/10.3389/fpsyg.2024.1221177
- The paper
- Chai, F., Ma, J., Wang, Y., Zhu, J., & Han, T. (2024). Grading by AI makes me feel fairer? How different evaluators affect college students’ perception of fairness. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2024.1221177
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.