CoChat
Sign Up
Cited by 9Artificial Intelligence in EducationEthics and Social Impacts of AIArtificial Intelligence in Healthcare and Education

Towards Responsible AI in Education: A Delphi-AHP-Based Framework for Evaluating Educational Large Language Models

Open access
Lin, Pingrong, Deng, Qin, Zhou, Yanbian·2026·Computers and Education: Artificial Intelligence
DOI 10.1016/j.caeai.2025.100534
Open this paper in CoChat — free account

Abstract

As large language models (LLMs) become deeply integrated into the educational landscape, evaluation criteria focusing solely on performance are insufficient to mitigate the risks of value misalignment and socioethical concerns. To steer educational LLMs towards responsible and beneficial development, this study aims to construct a multidimensional evaluation framework grounded in educational theory. Initially, a preliminary pool of evaluation indicators was established on the basis of a review of the literature and pedagogical theories. The Delphi method was subsequently employed to refine the indicator structure by integrating opinions from 21 cross-disciplinary experts. The analytic hierarchy process (AHP) was then applied to weigh these indicators and determine their priorities. The final framework comprises 5 first-level indicators and 21 s-level indicators. Learning effectiveness, knowledge construction capability, and social alignment are assigned critical weights, whereas intelligent interaction capability is less prioritized. Among the second-level indicators, information veracity was weighted the highest, while educational equity had the weakest influence. This study not only provides direction for the development and optimization of educational LLMs but also offers a reference for establishing a responsible artificial intelligence in education ecosystem.

Integrity signals

Registry

Crossref · 10.1016/j.caeai.2025.100534

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Only a small number of studies have applied structured, multi-criteria methodological frameworks for evaluation of the pedagogical suitability of LLM responses within specific educational domains.

    “Only a small number of studies have applied structured, multi-criteria methodological frameworks for evaluation of the pedagogical suitability of LLM responses within specific educational domains [15].”

    From An Integrated AHP–Fuzzy AHP Evaluation Framework for Large Language Models in Software Engineering Education (2026) · cites this paper as [15]

  • The problem of evaluating the quality of LLM-generated educational content using fuzzy multi-criteria decision-making methods remains insufficiently explored.

    “While the Delphi-AHP approach has been used to evaluate AI systems in education [15], the problem of evaluating the quality of LLMgenerated educational content using fuzzy multi-criteria decision-making methods remains insufficiently explored.”

    From An Integrated AHP–Fuzzy AHP Evaluation Framework for Large Language Models in Software Engineering Education (2026) · cites this paper as [15]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Lin, P., Deng, Q., & Zhou, Y.
Affiliations
Guangzhou University (CN)
Corresponding author
Lin, Pingrong
ORCID
Venue
Computers and Education: Artificial Intelligence
Publisher
Elsevier BV
Published
2026-06
Volume
10
Pages
100534
ISSN
2666-920X (ISSN-L)
DOI
10.1016/j.caeai.2025.100534
OpenAlex
W4417332316
Cited by
4
References
34
Citations by year
2026: 4
FWCI
4.19
Type
journal article
Language
en
Open access
Licence
Locations
Computers and Education Artificial Intelligence (publishedVersion, cc-by)
Funders
  • Guangzhou Office of Philosophy and Social Science
  • '13th Five-Year' Plan of Philosophy and Social Science Development in Guangzhou
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
responsible AI, LLM evaluation framework, evaluation metrics, value alignment, expert consensus, indicator weighting, learning effectiveness, knowledge construction, social alignment, information veracity, educational equity, intelligent interaction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review