CoChat
Sign Up
Cited by 1,806Artificial Intelligence in EducationArtificial Intelligence in Healthcare and EducationAcademic integrity and plagiarism

ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?

Open access
Jürgen Rudolph, Samson Tan, Shannon Tan·2023·Journal of Applied Learning & Teaching
DOI 10.37074/jalt.2023.6.1.9
Open this paper in CoChat — free accountOpen PDF

Abstract

ChatGPT is the world’s most advanced chatbot thus far. Unlike other chatbots, it can create impressive prose within seconds, and it has created much hype and doomsday predictions when it comes to student assessment in higher education and a host of other matters. ChatGPT is a state-of-the-art language model (a variant of OpenAI’s Generative Pretrained Transformer (GPT) language model) designed to generate text that can be indistinguishable from text written by humans. It can engage in conversation with users in a seemingly natural and intuitive way. In this article, we briefly tell the story of OpenAI, the organisation behind ChatGPT. We highlight the fundamental change from a not-for-profit organisation to a commercial business model. In terms of our methods, we conducted an extensive literature review and experimented with this artificial intelligence (AI) software. Our literature review shows our review to be amongst the first peer-reviewed academic journal articles to explore ChatGPT and its relevance for higher education (especially assessment, learning and teaching). After a description of ChatGPT’s functionality and a summary of its strengths and limitations, we focus on the technology’s implications for higher education and discuss what is the future of learning, teaching and assessment in higher education in the context of AI chatbots such as ChatGPT. We position ChatGPT in the context of current Artificial Intelligence in Education (AIEd) research, discuss student-facing, teacher-facing and system-facing applications, and analyse opportunities and threats. We conclude the article with recommendations for students, teachers and higher education institutions. Many of them focus on assessment.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.37074/jalt.2023.6.1.9

Cites

84 of 84 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The advent of GenAI has unveiled a plethora of threats to higher education and the broader societal fabric, encompassing concerns over authorship, copyright infringement, and the potential for misuse in activities such as spamming, disseminating fake news, creating malware, and hacking.

    “The advent of GenAI has unveiled a plethora of threats to higher education and the broader societal fabric, encompassing concerns over authorship, copyright infringement, and the potential for misuse in activities such as spamming, disseminating fake news, creating malware, and hacking (Marcus, 2023;Rudolph et al., 2023aRudolph et al., , 2023b)).”

    From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Rudolph et al., 2023a

    The “Use of ChatGPT for education: Challenges” section describes students outsourcing assignments to ChatGPT, supporting the claim of a threat to higher education. The copy does not substantiate the attributed concerns about copyright infringement or misuse for spamming, fake news, malware, and hacking; searches for those terms found no relevant discussion. Page numbers were not available in the indexed passages. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • ChatGPT is considered by many as the most powerful AI tool ever created.

    “It is considered by many as the most powerful AI tool ever crated [9].”

    From New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution (2023) · cites this paper as [9]

    The Abstract (page unavailable) calls ChatGPT “the world's most advanced chatbot thus far,” and the Introduction describes widespread enthusiasm for it. That supports the idea that many regard it as exceptionally capable, but not the broader claim that it is the most powerful AI tool ever created. The OpenAI section applies “most powerful” specifically to GPT-3 as a language model, not to ChatGPT among all AI tools. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • While AI technology offers promising ability for personalized learning experiences, efficient assessment, and adaptive feedback, it also presents novel challenges to the dynamics of the traditional educational bond.

    “While AI technology offers promising ability for personalized learning experiences, efficient assessment, and adaptive feedback, it also presents novel challenges to the dynamics of the traditional educational bond [9].”

    From New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution (2023) · cites this paper as [9]

    The Student-facing AI applications section describes adaptive, personalised learning, and the Teacher-facing AI applications section describes automated assessment and feedback (page numbers unavailable). The paper also discusses challenges to teaching and conventional assessment, but the passages do not establish a specific challenge to the traditional teacher–student bond. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • The diverse learning trajectories observed illustrate how AI tools can facilitate non-linear, adaptive learning processes by enabling students to dynamically integrate new information with prior knowledge and external resources.

    “The diverse learning trajectories observed in these cases illustrate how AI tools can facilitate non-linear, adaptive learning processes by enabling students to dynamically integrate new information with prior knowledge and external resources [15, 16].”

    From Scaffolding Creativity: Integrating Generative AI Tools and Real-World Experiences in Business Education (2025) · cites this paper as [16]

    The Student-facing AI applications section (page unavailable) says AI can scaffold learning in adaptive, personalised ways and provide personalised tutoring. It does not describe non-linear learning processes or students dynamically integrating new information with prior knowledge and external resources. The trajectories observed in the citing paper's cases are the citing authors' own observation, not a finding attributed to this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • ChatGPT is more student-friendly and can provide more need-based assistance than other AWE tools.

    “It is more student-friendly and can provide more need-based assistance than other AWE tools, as suggested by Guo et al. (2022) and Rudolph et al. (2023).”

    From Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study (2024) · cites this paper as Rudolph et al. (2023)

    The Student-facing AI applications section says ChatGPT helps students with writing, and the Opportunities section describes it as a learning aid. The paper does not say it is more student-friendly or provides more need-based assistance than other AWE tools; its discussion of personalised assistance concerns intelligent tutoring systems generally. Page numbers are not available in our copy. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 16

  1. [11]11. Aydın (Ed.), Emerging computer technologies 2 (pp. 22-31). İzmir Akademi Dernegi.
  2. [15]15. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021, March). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 610-623).
  3. [30]30. Cotton, D. R., Cotton, P. A., & Shipway, J. R. (2023, January 10). Chatting and cheating. Ensuring academic integrity in the era of ChatGPT. Preprint. https://doi.org/10.35542/osf. io/mrz8h D'Agostino, S. (2022, October 26). Machines can craft essays. How should writing be taught now? Inside Higher Ed, https:// www.insidehighered.com/news/2022/10/26/machines-can- craft-essays-how-should-writing-be-taught-now · DOI
  4. [45]45. Gilson, A., Safranek, C., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2022). How well does ChatGPT do when taking the medical licensing exams? The Implications of large language models for medical education and knowledge assessment. Preprint. medRxiv.
  5. [59]59. Heyward, E. (2022, May 16). Using AI to write essays isn't cheating, student says. Canta, (5), pp. 6-7. https://www. canta.co.nz/newsarticle/115524?newsfeedId=1453013
  6. [60]60. hooks, b. (2010). Teaching critical thinking. Routledge.
  7. [61]61. Huh, S. (2023). Are ChatGPT's knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: A descriptive study. Journal of Educational Evaluation for Health Professions, 20(1), https://doi.org/10.3352/jeehp.2023 · DOI
  8. [62]62. Jiao, W., Wang, W., Huang, J., Wang, X., & Tu, Z. (2023, January 3). Is ChatGPT a good translator? A preliminary study. Preprint. https://wxjiao.github.io/downloads/tech_ chatgpt_arxiv.pdf
  9. [77]77. Kurzweil, R. (2005). The singularity is near. Viking Penguin.
  10. [79]79. Lim, V. F. (2022, December 16). ChatGPT raises uncomfortable questions about teaching and classroom learning. The Straits Times, https://www.straitstimes.com/opinion/need- to-review-literacy-assessment-in-the-age-of-chatgpt Liquid Ocelot (2023, January 2). ChatGPT-4, the newest and most advanced AI system, might prompt a major shift in the way we communicate. Medium, https://medium.com/ inkwater-atlas/chatgpt-4-the-newest-and-most-advanced- ai-system-might-prompt-a-major-shift-in-the-way-we- fd764f97212c
  11. [103]103. Nisar, S., & Aslam, M. S. (2023, January 14). Is ChatGPT a good tool for T&CM students in studying pharmacology? Preprint. https://ssrn.com/abstract=4324310
  12. [105]105. O'Connor, S. & ChatGPT (2023). Open artificial intelligence platforms in nursing education: tools for academic progress or abuse? Nurse Education in Practice, 66, 103537. https:// doi.org/10.1016/j.nepr.2022.1 · DOI
  13. [109]109. Parker, M., Rudolph, J., Shukaitis, S., & Tan, S. (2021). Strategic utopianism and the avoidance of dualisms: An interview with Martin Parker. Journal of Applied Learning and Teaching, 4(Sp. Iss. 1), 60-73. https://doi.org/10.37074/jalt.2021.4.s1. · DOI
  14. [115]115. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre- training. Preprint. https://cdn.openai.com/research-covers/ language-unsupervised/language_understanding_paper. pdf
  15. [120]120. M.Ed. dissertation, University of Adelaide].
  16. [162]162. Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence

Cited by

Sorted by citations

Related

Authors
Rudolph, J., Tan, S., & Tan, S.
ORCID
Venue
Journal of Applied Learning & Teaching
Publisher
Open Access Publishing Association
Published
2023-01-25
Volume
6
Issue
1
ISSN
2591-801X (ISSN-L)
DOI
10.37074/jalt.2023.6.1.9
OpenAlex
W4317910584
Semantic Scholar
9460f9fbc532416c8ec39591577032a5fa4a79b3
Cited by
1,806
References
84
Citations by year
2026: 258 · 2025: 481 · 2024: 648 · 2023: 418 · 2022: 1
FWCI
424.07
Type
journal article
Language
en
Open access
Licence
other-oa
Locations
Journal of Applied Learning & Teaching (publishedVersion, other-oa)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Safety Research › Academic integrity and plagiarism
Keywords
ChatGPT, generative AI, student assessment, higher education, AI chatbots, artificial intelligence in education, OpenAI, future of teaching and learning, academic integrity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review