The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?
Open accessAbstract
ChatGPT is the world’s most advanced chatbot thus far. Unlike other chatbots, it can create impressive prose within seconds, and it has created much hype and doomsday predictions when it comes to student assessment in higher education and a host of other matters. ChatGPT is a state-of-the-art language model (a variant of OpenAI’s Generative Pretrained Transformer (GPT) language model) designed to generate text that can be indistinguishable from text written by humans. It can engage in conversation with users in a seemingly natural and intuitive way. In this article, we briefly tell the story of OpenAI, the organisation behind ChatGPT. We highlight the fundamental change from a not-for-profit organisation to a commercial business model. In terms of our methods, we conducted an extensive literature review and experimented with this artificial intelligence (AI) software. Our literature review shows our review to be amongst the first peer-reviewed academic journal articles to explore ChatGPT and its relevance for higher education (especially assessment, learning and teaching). After a description of ChatGPT’s functionality and a summary of its strengths and limitations, we focus on the technology’s implications for higher education and discuss what is the future of learning, teaching and assessment in higher education in the context of AI chatbots such as ChatGPT. We position ChatGPT in the context of current Artificial Intelligence in Education (AIEd) research, discuss student-facing, teacher-facing and system-facing applications, and analyse opportunities and threats. We conclude the article with recommendations for students, teachers and higher education institutions. Many of them focus on assessment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The advent of GenAI has unveiled a plethora of threats to higher education and the broader societal fabric, encompassing concerns over authorship, copyright infringement, and the potential for misuse in activities such as spamming, disseminating fake news, creating malware, and hacking (Marcus, 2023;Rudolph et al., 2023aRudolph et al., , 2023b)).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023a
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It is considered by many as the most powerful AI tool ever crated [9].”
From New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution (2023) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While AI technology offers promising ability for personalized learning experiences, efficient assessment, and adaptive feedback, it also presents novel challenges to the dynamics of the traditional educational bond [9].”
From New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution (2023) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The diverse learning trajectories observed in these cases illustrate how AI tools can facilitate non-linear, adaptive learning processes by enabling students to dynamically integrate new information with prior knowledge and external resources [15, 16].”
From Scaffolding Creativity: Integrating Generative AI Tools and Real-World Experiences in Business Education (2025) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It is more student-friendly and can provide more need-based assistance than other AWE tools, as suggested by Guo et al. (2022) and Rudolph et al. (2023).”
From Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study (2024) · cites this paper as Rudolph et al. (2023)
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the other hand, the promotion of learner autonomy and peer collaboration are similar to findings in the AWE literature (Dai et al., 2023;Rudolph et al., 2023).”
From Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study (2024) · cites this paper as Rudolph et al., 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“GPT-3's training on a vast corpus of 45 terabytes of text, sourced from a variety of internet-based entities like Common Crawl and Wikipedia, set a precedent for the scale of data utilised in AI development (Brown et al., 2020;Rudolph et al., 2023a).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023a
The cited source says what the claim attributes to it.
In the citing paper
“They highlight the transformative potential of generative AI in revolutionizing academic practices but also caution about its ability to facilitate academic dishonesty, such as through generating test answers [25].”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [25]
The cited source says what the claim attributes to it.
In the citing paper
“They highlight the transformative potential of generative AI in revolutionizing academic practices but also caution about its ability to facilitate academic dishonesty, such as through generating test answers [25].”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [25]
In the citing paper
“However, concerns have been raised about ethical consideration and decreased reliability of the existing examinations.”
From Developing Medical Education Curriculum Reform Strategies to Address the Impact of Generative AI: Qualitative Study (2023) · cites this paper as [8]
In the citing paper
“On the other hand, the impact of GAI on education is disputed [6], with studies raising concerns about the ethical considerations of ChatGPT [7], evaluation practices [8], scientific integrity [9], and potential negative effects on students’ higher order thinking skills [10].”
From Developing Medical Education Curriculum Reform Strategies to Address the Impact of Generative AI: Qualitative Study (2023) · cites this paper as [8]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Searle, J. R. (1980) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00005756 · Cited by 6,954
Teaching for quality learning at university
Boulton‐Lewis, G. M. (2008) · Faculty of Education · Cited by 4,926
Kung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519
Superintelligence: Paths, Dangers, Strategies
Robert, C. (2017) · CHANCE · doi:10.1080/09332480.2017.1302723 · Cited by 1,756
What is assessment for learning
Wiliam, D. (2011) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2011.03.001 · Cited by 1,282
Homo Deus: A Brief History of Tomorrow
Harari, Y. N. (2015) · Cited by 1,267
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022) · doi:10.1101/2022.12.23.521610 · Cited by 1,099
Research works printed but not linked: 16
- [11]11. Aydın (Ed.), Emerging computer technologies 2 (pp. 22-31). İzmir Akademi Dernegi.
- [15]15. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021, March). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 610-623).
- [30]30. Cotton, D. R., Cotton, P. A., & Shipway, J. R. (2023, January 10). Chatting and cheating. Ensuring academic integrity in the era of ChatGPT. Preprint. https://doi.org/10.35542/osf. io/mrz8h D'Agostino, S. (2022, October 26). Machines can craft essays. How should writing be taught now? Inside Higher Ed, https:// www.insidehighered.com/news/2022/10/26/machines-can- craft-essays-how-should-writing-be-taught-now · DOI
- [45]45. Gilson, A., Safranek, C., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2022). How well does ChatGPT do when taking the medical licensing exams? The Implications of large language models for medical education and knowledge assessment. Preprint. medRxiv.
- [59]59. Heyward, E. (2022, May 16). Using AI to write essays isn't cheating, student says. Canta, (5), pp. 6-7. https://www. canta.co.nz/newsarticle/115524?newsfeedId=1453013
- [60]60. hooks, b. (2010). Teaching critical thinking. Routledge.
- [61]61. Huh, S. (2023). Are ChatGPT's knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: A descriptive study. Journal of Educational Evaluation for Health Professions, 20(1), https://doi.org/10.3352/jeehp.2023 · DOI
- [62]62. Jiao, W., Wang, W., Huang, J., Wang, X., & Tu, Z. (2023, January 3). Is ChatGPT a good translator? A preliminary study. Preprint. https://wxjiao.github.io/downloads/tech_ chatgpt_arxiv.pdf
- [77]77. Kurzweil, R. (2005). The singularity is near. Viking Penguin.
- [79]79. Lim, V. F. (2022, December 16). ChatGPT raises uncomfortable questions about teaching and classroom learning. The Straits Times, https://www.straitstimes.com/opinion/need- to-review-literacy-assessment-in-the-age-of-chatgpt Liquid Ocelot (2023, January 2). ChatGPT-4, the newest and most advanced AI system, might prompt a major shift in the way we communicate. Medium, https://medium.com/ inkwater-atlas/chatgpt-4-the-newest-and-most-advanced- ai-system-might-prompt-a-major-shift-in-the-way-we- fd764f97212c
- [103]103. Nisar, S., & Aslam, M. S. (2023, January 14). Is ChatGPT a good tool for T&CM students in studying pharmacology? Preprint. https://ssrn.com/abstract=4324310
- [105]105. O'Connor, S. & ChatGPT (2023). Open artificial intelligence platforms in nursing education: tools for academic progress or abuse? Nurse Education in Practice, 66, 103537. https:// doi.org/10.1016/j.nepr.2022.1 · DOI
- [109]109. Parker, M., Rudolph, J., Shukaitis, S., & Tan, S. (2021). Strategic utopianism and the avoidance of dualisms: An interview with Martin Parker. Journal of Applied Learning and Teaching, 4(Sp. Iss. 1), 60-73. https://doi.org/10.37074/jalt.2021.4.s1. · DOI
- [115]115. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre- training. Preprint. https://cdn.openai.com/research-covers/ language-unsupervised/language_understanding_paper. pdf
- [120]120. M.Ed. dissertation, University of Adelaide].
- [162]162. Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence
Cited by
Sorted by citations
Items 1,001–1,031 of 1,031
Rücker, M. T., Büchting, C., & Kosch, T. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757693 · Cited by 7
“Moreover, model hallucinations and biases, giving rise to misinformation, can be detrimental to learning as well [6, 38].”
Learning with Generative AI : Evaluating Acceptability of Fact‐Checking Digital Nudges
Lee, C. S., & Nguyen, T. M. C. (2025) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1328 · Cited by 4
“However, educators have expressed concerns about misinformation when learning with GenAI, leading to negative learning outcomes (Rudolph et al., 2023; Sattelmaier & Pawlowski, 2023).”
Alzubi, A. A. F., & Alelaiwi, A. S. (2025) · Discover Computing · doi:10.1007/s10791-025-09793-x · Cited by 15
“Moreover, educators can allocate more time to providing feedback and refining students' work, as outlined by Rudolph et al. [38].”
InfluentialLi, L. L., Mao, G., Zhang, L., & Ying, J. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13849-2 · Cited by 4
Prinsloo, L. (2026) · SN Social Sciences · doi:10.1007/s43545-026-01405-9 · Cited by 4
“” (Rudolph et al. 2023) explores the disruptive role of ChatGPT and similar AI technologies in higher education.”
Uygun, E., & Demir, B. (2026) · Discover Education · doi:10.1007/s44217-026-01504-y · Cited by 15
“As cautioned by Bai and Hu [9] and Rudolph et al., [70], insufficient critical scrutiny may risk the standardization of language outcomes unless AI integration is guided by deliberate pedagogical design.”
Jacobs-Basadien, M., & Pather, S. (2026) · Universal Access in the Information Society · doi:10.1007/s10209-026-01340-9 · Cited by 11
Müller, Y., Lemberg, H., Gellert, P. R., & Zoellick, J. C. (2026) · JMIR Medical Education · doi:10.2196/88092 · Cited by 5
“…parrots” [10], might induce psychosis in vulnerable populations [11,12]: they “hallucinate” (ie, spread false information as credible) [13], reduce social interactions in professional relationships [14], or make established assessments such as essays insufficient in educational settings [15].”
The Game-Changing Impact of AI Chatbots on Education ChatGPT and Beyond
Vasudevan, A., Lama, A. V., & Sain, Z. H. (2024) · Journal of Information Systems and Technology Research · doi:10.55537/jistr.v3i1.770 · Cited by 4
Yurchenko, A. O., Shamonia, V., Буртовий, Р., Bohoslavskyi, S., & Семеніхіна, О. В. (2024) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro60963.2024.10569612 · Cited by 5
“Researchers identify the following problems: how to distinguish between reliable information and misinformation [10]; how to evaluate student work that is a product of AI [11]; what potential consequences will be an evaluation of AI-generated work [12]; how AI will affect scientific integrity [13].”
BackgroundAI Chatbots in Higher Education
Bolambao, F. J., Pogoy, A. M., & Plaisent, M. (2024) · Advances in higher education and professional development book series · doi:10.4018/979-8-3693-3904-6.ch003 · Cited by 4
Leveraging higher education for economic development
Chen, L., & Dong, X. (2025) · Finance research letters · doi:10.1016/j.frl.2025.107167 · Cited by 4
Quispe, D. J. M., Vera, E. A. L., Matos, M. M. H., & Barrios, J. P. M. (2025) · European Public & Social Innovation Review · doi:10.31637/epsir-2025-2015 · Cited by 8
“En los estudios previos se identificó que se demoró 10 años el implementar políticas educativas claras sobre los efectos de las herramientas de Inteligencia artificial (Rudolph et al ., 2023).”
Esfandiari, R., & Allaf-Akbary, O. (2025) · Innoeduca International Journal of Technology and Educational Innovation · doi:10.24310/ijtei.111.2025.20896 · Cited by 8
“However, it is widely accepted that AI-powered tools can strengthen language learning in terms of writing abilities (Kohnke et al., 2023; Rudolph et al., 2023).”
Cao, P., Khoshsaligheh, M., & Jomhouri, F. (2025) · Perspectives · doi:10.1080/0907676x.2025.2562024 · Cited by 5
de Oliveira, A. G., & Silveira, D. (2023) · Infarma - Ciências Farmacêuticas · doi:10.14450/2318-9312.v35.e1.a2023.pp3-5 · Cited by 4
Degrees of change: the promise of anti-racist assessment
Green, M. J., & Malcolm, C. (2023) · Frontiers in Sociology · doi:10.3389/fsoc.2023.972036 · Cited by 4
“The panic that ensued in HEIs regarding students using these tools was mostly, if not entirely, related to the fact that ChatGPT has the potential to disrupt traditional assessment practices in HE (Rudolph et al., 2023).”
BackgroundAI Tutor: Solution for China’s Disadvantaged and Under-resourced Children
Cao, B. (2023) · Lecture Notes in Education Psychology and Public Media · doi:10.54254/2753-7048/32/20230834 · Cited by 4
“Building on the GPT-3 foundation and enhanced through supervised and reinforcement learning, ChatGPT excels at engaging in human-like conversations and providing detailed responses to user queries [7].”
MethodDİNÇER, R., Dinçer, N., & GUKSU, O. (2023) · The Literacy Trek · doi:10.47216/literacytrek.1382543 · Cited by 5
Exploring Student Perspectives on ChatGPT in Higher Education
Shard, Vaidya, P., Koul, S., Arora, A., Sonia, & Jain, P. (2024) · 2024 3rd Edition of IEEE Delhi Section Flagship Conference (DELCON) · doi:10.1109/delcon64804.2024.10866660 · Cited by 3
Mann, R., Tosi, S., & Tree, D. (2025) · Emerging Topics in Life Sciences · doi:10.1042/etls20253021 · Cited by 3
“…disciplines, where practical skills and scientific reasoning are fundamental learning outcomes, the potential for AI to complete data analysis and written assessments/reports undermines the validity of these methods as evaluative tools, unless conducted under supervised conditions [2].”
Assessment design through co-creation
Williams, A., & Ingle, E. (2025) · International Journal for Students as Partners · doi:10.15173/ijsap.v9i1.5802 · Cited by 3
Lauren, P. (2024) · 2024 IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10893139 · Cited by 2
Guest Editorial Special Issue on Transforming Engineering Education
Lavi, R., Bertel, L. B., & Du, X. (2023) · IEEE Transactions on Education · doi:10.1109/te.2023.3308668 · Cited by 2
Dilber, C., & Yoşumaz, İ. (2026) · Journal of Academic Ethics · doi:10.1007/s10805-025-09681-5 · Cited by 1
Guoan, Z., Mengting, P., & Shaobo, W. (2025) · 2025 14th International Conference on Educational and Information Technology (ICEIT) · doi:10.1109/iceit64364.2025.10975838 · Cited by 1
ETHICAL USE OF ChatGPT IN RESEARCH WRITING
Namoco, S. O., & Ifedayo, A. E. (2026) · Veredas do Direito · doi:10.18623/rvd.v23.4903 · Cited by 1
Fotaris, P. (2026) · Cogent Arts & Humanities · doi:10.1080/23311983.2026.2722084
Diloy, M. A. (2025) · 2025 International Workshop on Artificial Intelligence and Education (WAIE) · doi:10.1109/waie67422.2025.11381028 · Cited by 1
From AI Literacy to AI Fluency: A Developmental Framework for Higher Education
Zentner, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7014818
Wang, J., Li, Y., & Qian, Y. (2025) · Proceedings of the 2025 9th International Conference on Education and Multimedia Technology · doi:10.1145/3761843.3761883
“Rudolph, Tan and Tan [9] describe a “new AI gold rush,” focusing on chatbots like Bard and ChatGPT, while their later work [10] questions ChatGPT’s impact on assessments.”
Metadata
- Authors
- Rudolph, J., Tan, S., & Tan, S.
- Venue
- Journal of Applied Learning & Teaching
- Publisher
- Open Access Publishing Association
- Published
- 2023-01-25
- Volume
- 6
- Issue
- 1
- ISSN
- 2591-801X (ISSN-L)
- OpenAlex
- W4317910584
- Semantic Scholar
- 9460f9fbc532416c8ec39591577032a5fa4a79b3
- Cited by
- 1,806
- References
- 84
- Citations by year
- 2026: 258 · 2025: 481 · 2024: 648 · 2023: 418
- FWCI
- 424.07
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- other-oa
- Locations
- Journal of Applied Learning & Teaching (publishedVersion, other-oa)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Education › Academic integrity and plagiarism
- Keywords
- ChatGPT, generative AI, student assessment, higher education, AI chatbots, artificial intelligence in education, OpenAI, future of teaching and learning, academic integrity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
