Cited by 120Ethics and Social Impacts of AIArtificial Intelligence in EducationIntelligent Tutoring Systems and Adaptive Learning

The life cycle of large language models in education: A framework for understanding sources of bias

Open access
Lee, Jinsook, Hicke, Yann, Yu, Renzhe, Brooks, Christopher, Kizilcec, René F.·2024·British Journal of Educational Technology

Abstract

Large language models (LLMs) are increasingly adopted in educational contexts to provide personalized support to students and teachers. The unprecedented capacity of LLM‐based applications to understand and generate natural language can potentially improve instructional effectiveness and learning outcomes, but the integration of LLMs in education technology has renewed concerns over algorithmic bias, which may exacerbate educational inequalities. Building on prior work that mapped the traditional machine learning life cycle, we provide a framework of the LLM life cycle from the initial development of LLMs to customizing pre‐trained models for various applications in educational settings. We explain each step in the LLM life cycle and identify potential sources of bias that may arise in the context of education. We discuss why current measures of bias from traditional machine learning fail to transfer to LLM‐generated text (eg, tutoring conversations) because text encodings are high‐dimensional, there can be multiple correct responses, and tailoring responses may be pedagogically desirable rather than unfair. The proposed framework clarifies the complex nature of bias in LLM applications and provides practical guidance for their evaluation to promote educational equity. Practitioner notes: What is already known about this topic The life cycle of traditional machine learning (ML) applications which focus on predicting labels is well understood. Biases are known to enter in traditional ML applications at various points in the life cycle, and methods to measure and mitigate these biases have been developed and tested. Large language models (LLMs) and other forms of generative artificial intelligence (GenAI) are increasingly adopted in education technologies (EdTech), but current evaluation approaches are not specific to the domain of education. What this paper adds A holistic perspective of the LLM life cycle with domain‐specific examples in education to highlight opportunities and challenges for incorporating natural language understanding (NLU) and natural language generation (NLG) into EdTech. Potential sources of bias are identified in each step of the LLM life cycle and discussed in the context of education. A framework for understanding where to expect potential harms of LLMs for students, teachers, and other users of GenAI technology in education, which can guide approaches to bias measurement and mitigation. Implications for practice and/or policy Education practitioners and policymakers should be aware that biases can originate from a multitude of steps in the LLM life cycle, and the life cycle perspective offers them a heuristic for asking technology developers to explain each step to assess the risk of bias. Measuring the biases of systems that use LLMs in education is more complex than with traditional ML, in large part because the evaluation of natural language generation is highly context‐dependent (eg, what counts as good feedback on an assignment varies). EdTech developers can play an important role in collecting and curating datasets for the evaluation and benchmarking of LLM applications moving forward.

Integrity signals

Registry

Crossref · 10.1111/bjet.13505

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Lee, J., Hicke, Y., Yu, R., Brooks, C., & Kizilcec, R. F.
Affiliations
  • Cornell University (US)
  • Columbia University (US)
  • University of Michigan (US)
Corresponding author
Lee, Jinsook
Venue
British Journal of Educational Technology
Publisher
Wiley
Published
2024-07-12
Volume
55
Issue
5
Pages
1982-2002
ISSN
  • 0007-1013 (ISSN-L)
  • 1467-8535
OpenAlex
W4400585758
Cited by
119
References
68
Citations by year
2026: 64 · 2025: 50 · 2024: 4 · 2023: 1
FWCI
49.32
Type
journal article
Language
en
Open access
Locations
  • British Journal of Educational Technology (publishedVersion)
  • Deep Blue (University of Michigan) (submittedVersion, other-oa)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
large language models, algorithmic bias, generative AI in education, educational technology, bias measurement, bias mitigation, educational equity, natural language generation evaluation, personalized tutoring

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review