Cited by 86Artificial Intelligence in EducationAcademic integrity and plagiarismArtificial Intelligence in Healthcare and Education

Is AI changing learning and assessment as we know it? Evidence from a ChatGPT experiment and a conceptual framework

Open access
Kolade, Oluwaseun, Owoseni, Adebowale, Egbetokun, Abiodun·2024·Heliyon

Abstract

ChatGPT, a state-of-the-art chatbot built upon Open AI's generative pre-trained transformer, has generated a major public interest and caused quite a stir in the higher education sector, where reactions have ranged from excitement to consternation. This paper therefore examines the potential impact of ChatGPT on learning and assessment, using the example of academic essays, being a major form of assessment with widespread applications of ChatGPT. This provides an opportunity to unpack broader insights on the challenge of generative AI's to the relevance, quality and credibility of higher education learning in a rapidly changing 21st century knowledge economy. We conducted a quasi-experiment in which we deployed ChatGPT to generate academic essays in response to a typical assessment brief, and then subjected the essays to plagiarism checks and independent grading. The results indicate that ChatGPT is able to generate highly original, and high quality, contents from distinct individual accounts in response to the same assessment brief. However, it is unable to generate multiple original contents from the same account, and it struggled with referencing. The discussion highlights the need for higher education providers to rethink their approach to assessment, in response to disruption precipitated by artificial intelligence. Thus, following the discussion of empirical data, we propose a new conceptual framework for AI-assisted assessment for lifelong learning, in which the parameters of assessment extend beyond knowledge (know what) testing, to competence (know how) assessment and performance (show how) evaluation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.heliyon.2024.e25953

Cites

48 of 51 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 52 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

  1. [5]Mao R. Chen G. Zhang X. Guerin F. Cambria E. GPTEval: A Survey on Assessments of ChatGPT and CHATGPT 2023 arXiv preprint arXiv:2308.12488 · arXiv
  2. [7]Brown T.B. Mann B. Ryder N. Subbiah M. Kaplan J. Dhariwal P. Neelakantan A. Shyam P. Sastry G. Askell A. Agarwal S. Herbert-Voss A. Krueger G. Henighan T. Child R. Ramesh A. Ziegler D.M. Wu J. Winter C. … Amodei D. Language Models Are Few-Shot Learners. 34th Conference on Neural Information Processing Systems (NeurIPS 2020) 2020 Vancouver, Canada https://commoncrawl.org/the-data/
  3. [26]Tack A. Piech C. The AI teacher test: measuring the pedagogical ability of blender and CHATGPT in educational dialogues arXiv preprint arXiv:2205.07540 2022 [Preprint] http://arxiv.org/abs/2205.07540 · arXiv
  4. [53]Yang A.C.M. Flanagan B. Ogata H. Adaptive formative assessment system based on computerized adaptive testing and the learning memory cycle for personalized learning Comput. Educ.: Artif. Intell. 3 2022 100104 10.1016/j.caeai.2022.100104 · DOI

Cited by

Sorted by citations

Authors
Kolade, O., Owoseni, A., & Egbetokun, A.
Affiliations
  • Sheffield Hallam University (GB)
  • De Montfort University (GB)
Corresponding author
Kolade, Oluwaseun
Venue
Heliyon
Publisher
Elsevier BV
Published
2024-02
Volume
10
Issue
4
Pages
e25953
ISSN
2405-8440 (ISSN-L)
OpenAlex
W4391715216
Cited by
66
References
52
Citations by year
2026: 23 · 2025: 29 · 2024: 14
FWCI
24.82
Type
journal article
Language
en
Open access
Locations
  • Heliyon (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • SHURA (Sheffield Hallam University Research Archive) (Sheffield Hallam University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • DMU Open Research Archive (De Montfort University) (submittedVersion, cc-by-nc-nd)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
ChatGPT, generative AI, AI-assisted assessment, academic essay writing, higher education assessment, plagiarism detection, AI-generated text quality, referencing, competency assessment, performance evaluation, lifelong learning, conceptual framework, knowledge economy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review