ChatGPT and Assessment in Higher Education: A Magic Wand or a Disruptor?
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
There is a current debate about the extent to which ChatGPT, a natural language AI chatbot, can disrupt processes in higher education settings. The chatbot is capable of not only answering queries in a human-like way within seconds but can also provide long tracts of texts which can be in the form of essays, emails, and coding. In this study, in the context of higher education settings, by adopting an experimental design approach, we applied ChatGPT-3 to a traditional form of assessment to determine its capabilities and limitations. Specifically, we tested its ability to produce an essay on a topic of our choice, created a rubric, and assessed the produced work in accordance with the designed rubric. We then evaluated the chatbot’s work by assessing ChatGPT’s application of its rubric according to a modified version of Paul’s (2005) Intellectual Standards rubric. Using Christensen et al.’s (2015) framework on disruptive innovations, our study found that ChatGPT was capable of completing the set tasks competently, quickly, and easily, like a “magic wand”. However, our findings also challenge the extent to which all of the ChatGPT’s demonstrated capabilities can disrupt this traditional form of assessment, given that there are aspects of its construction and evaluation that the technology is not yet able to replicate as a human expert would. These limitations of the chatbot can provide us with an opportunity for addressing vulnerabilities in traditional forms of assessment in higher education that are subject to academic integrity issues posed by this form of AI. We conclude the article with implications for teachers and higher education institutions by urging them to reconsider and revisit their practices when it comes to assessment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 3×
“The paper is expected to contribute to the sparse literature on the implications of the use of ChatGPT on higher education assessment, an issue which is receiving increasing attention in academia (Kasneci et al., 2023) and in educational practice (Stokel-Walker, 2022; Rudolph, Tan and Tan, 2023).”
BackgroundWhat if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00237-x · Cited by 2,003 · cited 1×
“This finding is in line with Pavlik (2023), Rudolph et al. (2023), and Tlili et al. (2023).”
ResultChatGPT: Bullshit spewer or the end of traditional assessments in higher education?
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.9 · Cited by 1,806 · cited 5×
“This finding is in line with Pavlik (2023), Rudolph et al. (2023), and Tlili et al. (2023).”
BackgroundResultInfluentialA SWOT analysis of ChatGPT: Implications for educational practice and research
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2195846 · Cited by 1,384 · cited 3×
“These findings r esonate with Farrokhnia et al. (2023), Rudolph, Tan and Tan (2023), and Qadir (2022) who also reported that the chatbot generated made-up reference lists with non-functional links.”
BackgroundBond, M., Buntins, K., Bedenlier, S., Zawacki-Richter, O., & Kerres, M. (2020) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0176-8 · Cited by 1,097 · cited 1×
“Studies about the role of technology in higher education have reported that it can enhance the learning process and impact student engagement (Bond et al., 2020) and improve communication between educators and students (Akour and Alenezi, 2022).”
BackgroundChatGPT and other large language models are double-edged swords
Shen, Y., Heacock, L., Elias, J., Hentel, K. D., Reig, B., Shih, G., & Moy, L. (2023) · Radiology · doi:10.1148/radiol.230163 · Cited by 1,008 · cited 2×
“Several studies investigating ChatGPT’s capabilities have highlighted its remarkable ability to produce a fluent piece of writing on a wide range of topics (Shen et al., 2023; Tate et al., 2022).”
BackgroundChatGPT User Experience: Implications for Education
Zhai, X. (2023) · doi:10.2139/ssrn.4312418 · Cited by 935 · cited 5×
“To date, many studies have focused on the chatbot’s potential to disrupt educational processes (Zhai, 2022), but there is still a dearth of research in identifying its gaps specifically in educational settings.”
BackgroundMethodInfluentialExploring the potential of using an AI language model for automated essay scoring
Mizumoto, A., & Eguchi, M. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100050 · Cited by 647
Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse?
O’Connor, S., & ChatGPT (2023) · Nurse Education in Practice · doi:10.1016/j.nepr.2022.103537 · Cited by 585
AI bot ChatGPT writes smart essays — should professors worry?
Stokel-Walker, C. (2022) · Nature · doi:10.1038/d41586-022-04397-7 · Cited by 543
Cited by
Evangelista, E. D. L. (2025) · Contemporary Educational Technology · doi:10.30935/cedtech/15775 · Cited by 124
“ZeroGPT AI detection tool focusing on identifying AI-written content (Klyshbekova et al., 2024).”
InfluentialChatGPT in Academia: University Students’ Attitudes Towards the use of ChatGPT and Plagiarism
Fajt, B., & Schiller, E. (2025) · Journal of Academic Ethics · doi:10.1007/s10805-025-09603-5 · Cited by 47
“Nevertheless, the integration of ChatGPT in education definitely raises ethical concerns about academic integrity, particularly in relation to plagiarism (Klyshbekova & Abbott, 2024 ).”
Kangwa, D., Msafiri, M. M., & Fute, A. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70109 · Cited by 22
Chiu, J., Castro, B., Ballard, I., Nelson, K., Zarutskie, P., Olaiya, O. K., Song, D., & Zhao, Y. (2025) · Medical Science Educator · doi:10.1007/s40670-025-02394-9 · Cited by 11
Exploring the Use of AI to Optimize the Evaluation of a Faculty Training Program
Míguez-Souto, A., Gutiérrez García, M. Á., & Martín-Núñez, J. L. (2025) · Education Sciences · doi:10.3390/educsci15101394 · Cited by 3
“Moreover, issues such as factual inaccuracies, citation errors, and inconsistency across runs have been well-documented (Klyshbekova & Abbott, 2024; Dengel et al., 2023).”
Metadata
- Authors
- Klyshbekova, M., & Abbott, P.
- Venue
- Electronic Journal of e-Learning
- Published
- 2024-02-09
- Pages
- 00-00
- ISSN
- 1479-4403 (ISSN-L)
- OpenAlex
- W4391676950
- Cited by
- 23
- References
- 20
- FWCI
- 8.80
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatGPT and Assessment in Higher Education: A Magic Wand or a Disruptor? https://app.cochat.ai/papers/10.34190/ejel.21.5.3114
- The paper
- Klyshbekova, M., & Abbott, P. (2024). ChatGPT and Assessment in Higher Education: A Magic Wand or a Disruptor? Electronic Journal of e-Learning. https://doi.org/10.34190/ejel.21.5.3114
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.