Cited by 994Ethics and Social Impacts of AIArtificial Intelligence in EducationOnline Learning and Analytics

Algorithmic bias in education

Open access
Baker, Ryan S., Hawn, Aaron·2022·International Journal of Artificial Intelligence in Education

Abstract

In this paper, we review algorithmic bias in education, discussing the causes of that bias and reviewing the empirical literature on the specific ways that algorithmic bias is known to have manifested in education. While other recent work has reviewed mathematical definitions of fairness and expanded algorithmic approaches to reducing bias, our review focuses instead on solidifying the current understanding of the concrete impacts of algorithmic bias in education—which groups are known to be impacted and which stages and agents in the development and deployment of educational algorithms are implicated. We discuss theoretical and formal perspectives on algorithmic bias, connect those perspectives to the machine learning pipeline, and review metrics for assessing bias. Next, we review the evidence around algorithmic bias in education, beginning with the most heavily-studied categories of race/ethnicity, gender, and nationality, and moving to the available evidence of bias for less-studied categories, such as socioeconomic status, disability, and military-connected status. Acknowledging the gaps in what has been studied, we propose a framework for moving from unknown bias to known bias and from fairness to equity. We discuss obstacles to addressing these challenges and propose four areas of effort for mitigating and resolving the problems of algorithmic bias in AIED systems and other educational technology.

Integrity signals

Registry

Crossref · 10.1007/s40593-021-00285-9

Cites

122 of 122 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • AI systems may replicate social and cultural inequalities embedded in training data, influencing assessment outcomes and learning opportunities.

    “Empirical studies show that AI systems may replicate social and cultural inequalities embedded in training data, influencing assessment outcomes and learning opportunities (Baker & Hawn, 2022;Barbierato et al., 2022;Deho et al., 2023;Jiang & Pardos, 2021).”

    From Academic Integrity and Students’ Ethical Use of ChatGPT in Higher Education (2026) · cites this paper as Baker & Hawn, 2022

  • ChatGPT's apparent fluency can encourage students to skip ethical consideration or critical examination, particularly when its biases remain hidden.

    “Similarly, ChatGPT's apparent fluency can encourage students to skip ethical consideration or critical examination, particularly when its biases remain hidden (Baker & Hawn, 2022).”
  • Algorithmic harms are often amplified in educational systems that rely heavily on predictive modelling for assessments, admissions, or instructional personalisation.

    “Their review shows that algorithmic harms are often amplified in educational systems that rely heavily on predictive modelling for assessments, admissions, or instructional personalisation.”
  • Automation may subtly displace critical judgment, especially in repetitive tasks or environments with ambiguous accountability structures.

    “Moreover, Baker and Hawn (2022) caution that automation may subtly displace critical judgment, especially in repetitive tasks or environments with ambiguous accountability structures.”
  • Students consistently identified the limitations of GenAI tools, whether factual errors in ChatGPT's market data or Eurocentric imagery generated by DALL•E, and responded by applying evaluative strategies.

    “Students consistently identified the limitations of GenAI tools, whether factual errors in ChatGPT's market data or Eurocentric imagery generated by DALL•E, and responded by applying evaluative strategies (Rana, 2024;Baker & Hawn, 2022).”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Baker, R. S., & Hawn, A.
Affiliations
University of Pennsylvania (US)
Corresponding author
Baker, Ryan S.
Venue
International Journal of Artificial Intelligence in Education
Publisher
Elsevier BV
Published
2022-12
Volume
32
Issue
4
Pages
1052-1092
ISSN
1560-4292 (ISSN-L)
OpenAlex
W3214911045
Cited by
993
References
122
Citations by year
2026: 393 · 2025: 276 · 2024: 165 · 2023: 96 · 2022: 59 · 2021: 4
FWCI
101.28
Type
journal article
Language
en
Open access
Locations
  • International Journal of Artificial Intelligence in Education (publishedVersion, cc-by-nc-nd)
  • EdArXiv (OSF Preprints) (submittedVersion)
Funders
  • Schmidt Futures
  • University of Pennsylvania
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
Keywords
algorithmic bias, artificial intelligence in education, algorithmic fairness, bias metrics, racial and ethnic bias, gender bias, nationality bias, socioeconomic status, disability, military-connected students, educational equity, bias mitigation, machine learning pipelines

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review