Cited by 839Academic integrity and plagiarismArtificial Intelligence in EducationArtificial Intelligence in Healthcare and Education

Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond

Open access
Perkins, Mike·2023·Journal of University Teaching and Learning Practice

Abstract

This paper explores the academic integrity considerations of students’ use of Artificial Intelligence (AI) tools using Large Language Models (LLMs) such as ChatGPT in formal assessments. We examine the evolution of these tools, and highlight the potential ways that LLMs can support in the education of students in digital writing and beyond, including the teaching of writing and composition, the possibilities of co-creation between humans and AI, supporting EFL learners, and improving Automated Writing Evaluations (AWE). We describe and demonstrate the potential that these tools have in creating original, coherent text that can avoid detection by existing technological methods of detection and trained academic staff alike, demonstrating a major academic integrity concern related to the use of these tools by students. Analysing the various issues related to academic integrity that LLMs raise for both Higher Education Institutions (HEIs) and students, we conclude that it is not the student use of any AI tools that defines whether plagiarism or a breach of academic integrity has occurred, but whether any use is made clear by the student. Deciding whether any particular use of LLMs by students can be defined as academic misconduct is determined by the academic integrity policies of any given HEI, which must be updated to consider how these tools will be used in future educational environments.

Integrity signals

Registry

Crossref · 10.53761/1.20.02.07

Cites

89 of 89 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

29 of 60 cited sources checked · 15 supported · 0 flagged · accuracy not yet measured

  • Premisecitation
    § Limitations and newer LLMs · p. 9

    LLMs have the potential to produce text that lacks semantic coherence.

    “there is the potential for LLMs to produce text that lacks semantic coherence (Dale, 2021)”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Dale, 2021 source
  • Premisecitation
    § Introduction · p. 3

    The pandemic resulted in increases in student or academic staff perceptions of academic dishonesty.

    “as well as increases in student or academic staff perceptions of AD (Amzalag et al., 2021; Reedy et al., 2021; Walsh et al., 2021) occurring.”
    Quote in the paperNot a substantial claim3 of 3 sources checked
    • cites Amzalag et al., 2021 source

      The Findings (page unavailable) report students’ and lecturers’ perceptions of academic dishonesty during pandemic-era online exams, including lecturers’ concern about copying. They compare the two groups, not perceptions before and during the pandemic, so they do not establish that the pandemic increased those perceptions. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Reedy et al., 2021 source
    • cites Walsh et al., 2021 source
  • Interpretationcitation
    § Student uses of LLMs · p. 10

    Student use of an LLM to support work, including directly copied output, without stating AI use, could be seen as a relatively clear case of plagiarism.

    “given the misrepresentation that have been made, usage of this type could be seen as a relatively clear case of plagiarism by the student”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Roe & Perkins, 2022 source

      In “Types of APTs and their use in academic work” (page unavailable), the authors call using automated paraphrasing tools to rework copied source material academic dishonesty and paraphrasing plagiarism. That supports the general concern about presenting machine-assisted work as one’s own, but the paper does not address LLM-generated work or establish that directly copying undisclosed LLM output is a clear case of plagiarism. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Premisecitation
    § Concerns for academic integrity · p. 9

    Use of LLM-created text by students is unlikely to be spotted by existing text-matching software tools used by HEIs.

    “current research suggests that use of the created text by students is unlikely to be spotted by existing text-matching software tools used by HEIs”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Wahle, Ruas, Kirstein, et al., 2022 source

      In Q5, the paper reports that PlagScan could not reliably identify machine-paraphrased text and that large language models made text matches harder to find. Its tests concern LLM-generated paraphrases and one commercial plagiarism-detection product, not all LLM-created text or the range of text-matching tools used by higher education institutions. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • Rests on an assumption: LLM-generated text is uniquely created based on the inputs provided.
  • Premisecitation
    § Automated Writing Evaluation (AWE) · p. 14

    Use of LLMs in AWE tools can reduce the burden on EFL instructors.

    “the use of LLMs in these tools can particularly reduce the burden on EFL instructors (Godwin-Jones, 2022).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Godwin-Jones, 2022 source

      The Ecological Perspectives section (page unavailable) says AWE can free teachers’ time from correcting language mechanics so they can address higher-order writing issues. The Introduction describes large language models as enabling language services, but the paper does not specifically say that LLMs in AWE tools reduce instructors’ workload, or that the benefit is particular to EFL instructors. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Classified as probably not a substantial claim. 47
  • May combine two claims. 18
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § The emergence of Large Language Models · p. 5

    Transformer based machine learning models allow for improved performance on language based tasks when compared to previous machine learning techniques.

    “These tools started to achieve prominence from 2017 with the emergence of a range of transformer based machine learning models which allow for improved performance on language based tasks when compared to previous machine learning techniques (Vaswani et al., 2017).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Vaswani et al., 2017 source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § The evolution of AI powered digital writing tools · p. 4

    Digital Writing Assistants (DWAs) such as Grammarly, WordTune and Perusall use Artificial Intelligence (AI) to improve student writing.

    “Digital Writing Assistants (DWAs) such as Grammarly, WordTune and Perusall, use Artificial Intelligence (AI) to improve student writing (Fitria, 2021)”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Fitria, 2021 source

      The Abstract (page not specified) describes Grammarly as an AI-powered writing assistant and reports improved writing performance after its use. The paper supports that part of the claim, but the passages found for WordTune and Perusall do not address either tool, so it does not substantiate the claim about all three. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § The evolution of AI powered digital writing tools · p. 4

    DWAs have demonstrated a particular ability to support English as a Foreign Language (EFL) writers in improving their skills at expressing ideas in written English.

    “These tools have also demonstrated a particular ability to support English as a Foreign Language (EFL) writers in improving their skills at expressing ideas in written English (Gayed et al., 2022; Nazari et al., 2021; Zhao, 2022)”
    Quote in the paperNot a substantial claim3 of 3 sources checked
    • cites Gayed et al., 2022 source

      The Results section reports that adult EFL participants using AI KAKU produced sentences with significantly greater syntactic complexity, which the authors link to communicating complex ideas more fluently. But the study tested performance during a writing task, not lasting improvement in writing skills or a particular ability of digital writing aids generally. Differences in word production and lexical diversity were not statistically significant. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Nazari et al., 2021 source

      The Discussion says Grammarly can improve EFL writing skills, and the Conclusion describes it as an assistive tool for non-native postgraduate students writing in English. But the trial reports improvements in writing self-efficacy, engagement, and emotions—not a measured improvement in students’ ability to express ideas in writing. The Introduction’s reference to expressing ideas describes writing skills generally, not a demonstrated effect of the tool. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites Zhao, 2022 source
  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Premisecitation
    § Limitations and newer LLMs · p. 9

    LLMs will generally always provide an answer or output, even if that is factually incorrect or misinterpreted.

    “Although one concern levied against LLMs is that they will generally always provide an answer or output, even if that is factually incorrect or misinterpreted (Dale, 2021; Hutson, 2022)”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Dale, 2021 source

      Section 3.3, “From editor to author” (page unavailable), says GPT-3 can generate convincing text that is sometimes nonsense, supporting the concern that an output can be unreliable. It does not say LLMs generally always provide an answer, or specifically discuss factual errors or misinterpretation. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites Hutson, 2022 source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § Supporting EFL learners · p. 13

    The GPT-2 based tool could support improving lexical diversity of student work, but statistically significant results were not obtained.

    “Their study suggested that this tool could support in improving the lexical diversity of student work, but statistically significant results were not obtained.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Gayed et al., (2022) source
  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation
    § Practitioner Notes · p. 2

    There are legitimate uses of AI tools using LLMs in supporting the education of students.

    “There are legitimate uses of these tools in supporting the education of students, meaning HEIs must carefully consider how policies dealing with student use of this software are created.”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites Chen et al., 2015 source
    • cites Park & Yang, 2020

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Zhao, 2022 source

      The Affordance for English Writing section (p. 4) says Wordtune offers learning opportunities for EFL writers and suggests that teachers encourage students to use it in writing activities. This supports an educational use of an AI tool, but our copy does not identify Wordtune as using an LLM. The sentence’s policy recommendation is the citing authors’ inference, not a point attributed to this source. (LLM, conf=0.79)

      Experimental — accuracy not yet measured
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Technological methods of detection · p. 15

    Detectors for machine-created text may potentially discriminate against EFL students by incorrectly identifying human created text as machine written.

    “highlight the ethical challenges of deploying any such detectors which may potentially discriminate against EFL students by incorrectly identifying human created text as machine written”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Fröhling & Zubiaga (2021) source
  • May combine two claims.

    Study design and conductcitation
    § Technological methods of detection · p. 15

    GLTR was tested against GPT-2 produced output and used students rather than academic staff as participants.

    “However, this tool was tested against GPT-2 produced output and used students rather than academic staff as their participants.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Gehrmann et al. (2019) source
  • May combine two claims.

    Premisecitation
    § Technological methods of detection · p. 15

    Tools by Solaiman et al. (2019) and Ippolito et al. (2020) showed an encouraging ability in detecting machine created text, but were not tested against the latest generation of LLMs.

    “Solaiman et al. (2019) and Ippolito et al.(2020) also both present tools which showed an encouraging ability in detecting machine created text, although these tools were not tested against the latest generation of LLMs.”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites Solaiman et al. (2019) source
    • cites Ippolito et al.(2020) source
  • Classified as probably not a substantial claim.

    Findingnone given
    § References: · p. 7

    The in-text citations and references provided by ChatGPT in the example were all fabricated, with titles, URLs, and journal names being incorrect.

    “Although in text citations and references were provided (including references from the input text), these were all fabricated, with titles, URLs, and journal names being incorrect.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation
    § References: · p. 8

    The references in the second ChatGPT output were falsified and there were some inaccuracies related to the claimed ability of Turnitin to detect AI generated content.

    “Again, the references were falsified and there were some inaccuracies related to the claimed ability of Turnitin to detect AI generated content, but this fluent, five-paragraph output is typical of a response by ChatGPT.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given
    § References: · p. 7

    A major benefit of ChatGPT as opposed to previous versions of the GPT-3 software is the iteration possible through natural language 'discussions' held with the software.

    “A major benefit of ChatGPT as opposed to previous versions of the GPT-3 software is that of the iteration possible through natural language 'discussions' held with the software.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given
    § References: · p. 7

    The output retained the language style used in the input and expanded on the discussions by providing examples specifically related to education as requested.

    “The output retained the language style used in the input and expanded on the discussions by providing examples specifically related to education as requested.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given
    § Practitioner Notes · p. 2

    AI tools using LLMs can be used to create original written content that students may use in their assessments.

    “These tools use Large Language Models (LLMs) and can be used to create original written content that students may use in their assessments.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given
    § Limitations and newer LLMs · p. 9

    ChatGPT can be challenged if the user believes there are incorrect elements in the output, and can adjust its response accordingly based on any corrections provided by the user.

    “ChatGPT in particular can also be challenged if the user believes there are incorrect elements in the output, and can adjust its response accordingly based on any corrections provided by the user.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation
    § References: · p. 7

    The output created by ChatGPT in the paper's example was created in approximately 5 seconds at zero cost.

    “This output was created in approximately 5 seconds at zero cost (ChatGPT is currently in a free research preview stage and costs are likely to change in the near future).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Findingnone given
    § References: · p. 8

    The fluent, five-paragraph output is typical of a response by ChatGPT.

    “but this fluent, five-paragraph output is typical of a response by ChatGPT.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § Abstract · p. 2

    Whether any particular use of LLMs by students can be defined as academic misconduct is determined by the academic integrity policies of any given Higher Education Institution (HEI).

    “Deciding whether any particular use of LLMs by students can be defined as academic misconduct is determined by the academic integrity policies of any given HEI, which must be updated to consider how these tools will be used in future educational environments.”
    • Also stated in: § Practitioner Notes · p. 2 · hedged
      “however it may be a breach of academic integrity policies of any given Higher Education Institution (HEI).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given
    § Abstract · p. 2

    Whether plagiarism or a breach of academic integrity has occurred is not defined by the student's use of any AI tools, but by whether the student makes any use clear.

    “we conclude that it is not the student use of any AI tools that defines whether plagiarism or a breach of academic integrity has occurred, but whether any use is made clear by the student.”
    • Also stated in: § Practitioner Notes · p. 2 · hedged
      “The use of these tools may not necessarily be considered as plagiarism if students are transparent in how they have been used in any submission, however it may be a breach of academic integrity policies of any given Higher Education Institution (HEI).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning
    § Plagiarism or Academic Misconduct? · p. 11

    Use of LLMs or LLM-based tools does not constitute plagiarism or a breach of academic integrity if the use is stated clearly and made transparent by students.

    “we conclude that the use of LLMs or LLM based tools should not be considered as plagiarism or a breach of academic integrity if how these tools have been used is stated clearly and made transparent by students.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § References: · p. 7

    Fabricated references open a potential avenue of detection by academic staff if there is a suspicion that these tools may have been used by students.

    “This does open a potential avenue of detection by academic staff if there is a suspicion that these tools may have been used by students.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § Practitioner Notes · p. 2

    LLM-based tools can be accessed using commercial services built on this software, often targeted to students as a means of 'assisting' students with assessments.

    “These tools can be accessed using commercial services built on this software, often targeted to students as a means of 'assisting' students with assessments.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § Integration of LLMs into commercial services · p. 6

    Many commercial tools that provide access to LLMs integrate features of DWAs and APTs, and market the product package together as one 'writing support tool'.

    “Many of these tools integrate features of DWAs and APTs, and market the product package together as one 'writing support tool' or similar, available as both free and paid for versions.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § Cognitive offloading · p. 11

    It is feasible that a student could very easily and inadvertently breach policies related to correct usage of LLMs, especially when LLMs become even more deeply integrated into DWAs.

    “it is feasible to believe that a student could very easily and inadvertently breach any policies related to their 'correct' usage, especially when LLMs become even more deeply integrated into DWAs.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Many advanced DWAs have multiple features available to students which already claim to integrate AI technologies.
    • Rests on an assumption: Write Full is a full-service DWA which has integrated paraphrasing tools that may encourage students to use the work of others as inputs, rather than their own writing.
  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning
    § Student uses of LLMs · p. 10

    Student use of an LLM to create an entire essay, report, or assignment without stating its origin, or with false citations to mislead, is clearly plagiarism and likely a broader breach of academic integrity principles.

    “Student usage of an LLM or LLM based tool to create an entire essay, report, or other assignment without stating where this has come from, or one that is presented in a way that was trying to mislead the reader through false citations, would be even more clearly a case of plagiarism, and likely also a broader breach of academic integrity principles.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § The emergence of Large Language Models · p. 5

    Large Language Models (LLMs) are able to develop significant amounts of brand-new text based on short input requests.

    “Tools using Artificial Intelligence (AI) to support in the creation of original text have emerged in recent years, in the form of Large Language Models (LLMs) which are able to develop significant amounts of brand-new text based on short input requests.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § References: · p. 7

    Given that the authors and dates provided in the in-text citations are correct, fabricated references may go unnoticed unless thoroughly checked.

    “However, given that the authors and dates provided in the intext citations are correct, this may go unnoticed unless thoroughly checked.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning
    § Student uses of LLMs · p. 10

    If a student's work has not been misrepresented, it does not fall under the strict definition of plagiarism, but it may be considered academic misconduct depending on specific HEI policies.

    “If the work carried out by a student hasn't been misrepresented, then it does not fall under the strict definition of plagiarism, but it may be considered as academic misconduct depending on the specific policies and regulations in place at individual HEIs.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § Integration of LLMs into commercial services · p. 6

    A subset of these tools is marketed specifically to students to support in the development of full length-assessments, which also include options to generate citations alongside the work.

    “A subset of these tools is marketed specifically to students to support in the development of full length-assessments, which also include options to generate citations alongside the work.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § The emergence of Large Language Models · p. 5

    ChatGPT achieved more than 1 million users within five days of launch.

    “as well as achieving more than 1 million users within five days of launch (Brockman, 2022).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Brockman, 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning
    § Plagiarism or Academic Misconduct? · p. 11

    There is a potential danger of learning outcomes not being met if LLMs are used to develop content rather than content being created by students themselves.

    “HEIs must be aware of the potential danger of learning outcomes not being met if LLMs are being used to develop content, rather than this being created by students themselves.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Creative outputs · p. 12

    Dall-E 2 can create and edit detailed digital artwork based on natural language prompts provided by users.

    “Dall·E 2 AI system which is able to create and edit detailed digital artwork based on natural language prompts provided by users”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites OpenAi, n.d.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § References: · p. 8

    With some manipulation of the text through iteration using the tool, along with some minor editing or adjustments of references, a student intending to create large blocks of text to include in an assessment could rapidly complete certain tasks with limited effort and costs.

    “It is evident that with some manipulation of the text through iteration using the tool, along with some minor editing or adjustments of references, a student intending to create large blocks of text to include in an assessment could rapidly complete certain tasks with limited effort and costs.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The ChatGPT output generated in the paper's demonstration is representative of LLM output generally.
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning
    § Integration of LLMs into commercial services · p. 6

    LLMs can be accessed through commercial entities that use an Application Programming Interface (API) to connect to the LLMs, and these services are targeted at different groups of customers including students.

    “an alternative method that students may use to access the LLMs is through commercial entities who use an Application Programming Interface (API) to connect to the LLMs. These services are targeted at different groups of customers including copywriters, marketers and students and allow user-friendly access to the LLMs.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § Practitioner Notes · p. 2

    Students now have easy access to advanced Artificial Intelligence based tools such as ChatGPT.

    “Students now have easy access to advanced Artificial Intelligence based tools such as ChatGPT.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation
    § Automated Writing Evaluation (AWE) · p. 14

    Present AWE tools have shortcomings: generic feedback and inability to focus feedback on how well a task has been addressed.

    “Some of the recognised shortcomings of present AWE tools are the generic nature of the feedback provided (Fu et al., 2022) and an inability to focus feedback on how well a task has been addressed (Allen et al., 2016).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Fu et al., 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Allen et al., 2016

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § Concerns for academic integrity · p. 9

    LLMs represent a clear potential threat to academic integrity because academic staff may be unable to correctly identify the amount of content produced by a student and provide an accurate evaluation of the student's comprehension and interpretation.

    “LLMs therefore represent a clear potential threat to academic integrity as academic staff may be unable to correctly identify the amount of content produced by a student, and therefore provide an accurate evaluation of a student's comprehension and interpretation of the topic at hand.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Automated Writing Evaluation (AWE) · p. 13

    Software tools for automated writing evaluation (AWE) benefit writing instructors and students.

    “Using software tools to support in AWE of written text and providing feedback to students has been demonstrated to be of benefit to writing instructors and students alike”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Fu et al., 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Strobl et al., 2019 source
    • cites Warschauer & Grimes, 2008 source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § The emergence of Large Language Models · p. 5

    The broad release to the public in November 2022 of ChatGPT by OpenAI marked a significant increase in the ability of the underlying software to create new text using further refined models (GPT-3.5), coupled with a markedly improved user interface.

    “The broad release to the public in November 2022 of ChatGPT by OpenAI marked a significant increase in the ability of the underlying software to create new text using further refined models (GPT-3.5), coupled with a markedly improved user interface (OpenAI, 2022).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites OpenAI, 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Writing and composition · p. 12

    Using LLMs in the teaching of digital writing can support writers in engaging more deeply with a topic, rather than at a surface level.

    “Using LLMs in the teaching of digital writing therefore becomes a method to support writers in engaging more deeply with a topic, rather than at a surface level.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites McKnight (2021) source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Introduction · p. 3

    During the pandemic, students sought out and adapted to new technologies.

    “During the period of the pandemic, research has shown that students sought out and adapted to new technologies (Vargo et al., 2021)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Vargo et al., 2021 source
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation
    § Conclusion · p. 17

    It is unlikely that AI will ever take over the traditional role of a teacher.

    “Although it is unlikely that AI will ever take over the traditional role of a teacher (Cope et al., 2021)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Cope et al., 2021 source
  • May combine two claims.

    Premisecitation
    § GPT-2 studies · p. 14

    Training academic staff increased their ability to correctly identify text sections, but the samples were generated using GPT-2 rather than GPT-3.

    “Although it is encouraging to note that training of academic staff resulted in an increased ability of academic staff to correctly identify the text sections, the samples were generated using GPT-2 as opposed to GPT-3.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Abd-Elaal et al., (2022) source
  • May combine two claims.

    Interpretationreasoning
    § Conclusion · p. 17

    The likelihood of accurately detecting any usage of LLMs by students in their submissions will likely not improve and may even decrease further as new LLMs are developed.

    “the likelihood of accurately detecting any usage of these tools by students in their submissions typical academic will likely not improve and may even decrease further as new LLMs are developed.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: New large language models will continue to be developed.
  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning
    § Conclusion · p. 16

    Whether plagiarism or a breach of academic integrity has occurred is defined not by the use of the tools themselves but by whether any such use is made clear.

    “it is not the use of the tools themselves that defines whether plagiarism or a breach of academic integrity has occurred, but whether any such use is made clear.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation
    § Conclusion · p. 16

    What academic staff define as plagiarism is rapidly changing due to the social construction of the term.

    “What we as academic staff define as plagiarism is rapidly changing due to the social construction of this term (Anson, 2022).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Anson, 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § Technological methods of detection · p. 15

    GPTZero and Crossplag AI detect show promise in being able to detect the use of AI generated text.

    “These tools include GPTZero (https://gptzero.me/) and Crossplag AI detect (https://crossplag.com/ai-content-detector/) and do show promise in being able to detect the use of AI generated text.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites GPTZero (https://gptzero.me/)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Crossplag AI detect (https://crossplag.com/ai-content-detector/)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Conclusion · p. 16

    Plagiarism overall decreased between 1990 and 2020.

    “Although evidence suggests that plagiarism overall has decreased between 1990 and 2020 (Curtis, 2022)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Curtis, 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning
    § Conclusion · p. 17

    A blanket ban of LLM-based tools at an institutional level is neither feasible nor enforceable.

    “a blanket ban of these tools at an institutional level is neither feasible, nor enforceable.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given
    § Conclusion · p. 16

    Whether any particular use of LLMs by students is defined as academic misconduct will be determined by the future policies of any given HEI.

    “Deciding whether any particular use of LLMs by students may be defined as academic misconduct will be determined by the future policies of any given HEI”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: HEIs will set future policies governing LLM use.
  • May combine two claims.

    Premisecitation
    § GPT-3 studies · p. 14

    In Kumar et al. (2022), participants regardless of background found identifying GPT-3 text challenging, with a high likelihood of ascribing AI writing samples to humans.

    “They found that regardless of background, participants found this to be a challenging task, with a high likelihood of ascribing the AI writing samples to humans.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kumar et al. (2022)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation
    § Conclusion · p. 17

    The future development of LLMs and broader AI supported digital tools has a strong potential for improving the experiences of students and teachers alike in the next generation of HEI classrooms, both in writing instruction and beyond.

    “the future development of LLMs and broader AI supported digital tools have a strong potential for improving the experiences of students and teachers alike in the next generation of HEI classrooms, both in writing instruction and beyond.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning
    § Technological methods of detection · p. 16

    Both academic staff and technological methods of detection are unable to accurately detect machine generated text and therefore student uses of LLM based tools.

    “Given that both academic staff, as well as technological methods of detection are unable to accurately detect machine generated text and therefore student uses of LLM based tools”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning
    § Conclusion · p. 17

    The future integration of LLMs and other AI supported digital tools into the classroom environment is highly likely.

    “we believe that the future integration of LLMs and other AI supported digital tools into the classroom environment is highly likely”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § Conclusion · p. 16

    The inability to accurately detect whether LLMs have been used presents a clear threat to the academic integrity of HEIs.

    “the inability to accurately detect whether LLMs have been used presents a clear threat to the academic integrity of HEIs.”
    Quote in the paperNot a substantial claim
55 of 55
  • The paper relies on this without support.

    The ChatGPT output generated in the paper's demonstration is representative of LLM output generally.

    “As an example of the fluency of the output that can be created using these tools, the two paragraphs of the section above titled: 'The emergence of Large Language Models' were used as an input to ChatGPT.”
  • The paper relies on this without support.

    LLM-generated text is uniquely created based on the inputs provided.

    “given that the text that is produced by LLMs is uniquely created based on the inputs provided”
  • The paper relies on this without support.

    Many advanced DWAs have multiple features available to students which already claim to integrate AI technologies.

    “Given that many of the more advanced DWAs have multiple features available to students which already claim to integrate AI technologies”
  • The paper relies on this without support.

    Write Full is a full-service DWA which has integrated paraphrasing tools that may encourage students to use the work of others as inputs, rather than their own writing.

    “This already appears to be the case in the example of one DWA called Write Full: a full-service DWA which has integrated paraphrasing tools which may encourage students to use the work of others as inputs, rather than their own writing.”
  • The paper relies on this without support.

    New large language models will continue to be developed.

    “as new LLMs are developed.”
  • The paper relies on this without support.

    HEIs will set future policies governing LLM use.

    “will be determined by the future policies of any given HEI”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Universities have adopted varied approaches to generative AI: some have banned it or employed detection tools, while others permit and even encourage its use with appropriate acknowledgment.

    “In response, universities have adopted varied approaches: some have banned generative AI or employed detection tools, while others permit and even encourage its use with appropriate acknowledgment (Perkins 2023; Sullivan, Kelly, and McLaughlan 2023; Xiao, Chen, and Bao 2023).”

    From Addressing student non-compliance in AI use declarations: implications for academic integrity and assessment in higher education (2025) · cites this paper as Perkins 2023

    The Practitioner Notes (p. 2) say disclosed AI use may not be plagiarism, and Academic integrity policy adjustments (p. 16) recommends a nuanced approach rather than a blanket ban. The paper discusses bans, detection, and disclosure as policy considerations, but the cited passages do not establish that universities have already adopted the varied approaches described. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • Initially, the use of GenAI tools in academic settings was largely viewed as a form of plagiarism or academic misconduct.

    “Initially, the use of GenAI tools in academic settings was largely viewed as a form of plagiarism or academic misconduct (Cotton et al., 2023; Perkins, 2023).”

    From The Artificial Intelligence Assessment Scale (AIAS): A Framework for Ethical Integration of Generative AI in Educational Assessment (2024) · cites this paper as Perkins, 2023

    The Practitioner Notes (p. 2) identify potential plagiarism and academic-integrity breaches from student use of these tools, so the paper supports the concern underlying the claim. But the same passage says transparent use need not be plagiarism, and the discussion of acceptable use (p. 13) argues against a blanket ban. The paper does not establish that GenAI use was initially or largely viewed as misconduct. (LLM, conf=0.85)

    Experimental — accuracy not yet measured
  • Banning or blocking GenAI tools has proven ineffective.

    “Banning or blocking GenAI tools has proven ineffective, and punitive approaches ignore the potential benefits of these technologies.”

    From The AI Assessment Scale (AIAS) in action: A pilot implementation of GenAI supported assessment (2024) · cites this paper as Perkins, 2023

    The Academic integrity policy adjustments section (p. 16) says a blanket ban is likely unenforceable, and the Conclusion (p. 17) calls an institutional blanket ban neither feasible nor enforceable. These passages support the concern about banning, but they do not establish that bans or technical blocking have already proven ineffective. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • This focus leads to a narrow perception of AI as only a source of academic malpractice.

    “it leads to a narrow perception of AI as only a source of academic malpractice (Perkins, 2023).”

    From Assessment-focused governance mode? Mapping Generative AI policies across 76 UK higher education institutions: Policy framework and recommendations (2026) · cites this paper as Perkins, 2023

    The classroom-use section (p. 12) contrasts academic-integrity concerns with legitimate educational uses of AI, and the policy section (p. 16) calls for a nuanced approach that recognizes its benefits. These passages support the objection to viewing AI only as a source of malpractice, but they do not establish that a particular focus leads people to adopt that narrow perception. (LLM, conf=0.79)

    Experimental — accuracy not yet measured
  • The current study identified critical thinking, self-reflection, and data interpretation as areas where students could work collaboratively with generative AI tools without impinging on institutional academic integrity policies.

    “The current study identified critical thinking, self-reflection, and data interpretation as areas where students could work collaboratively with generative AI tools without impinging on institutional academic integrity policies (Perkins, 2023).”

    From Assessment design through co-creation (2025) · cites this paper as Perkins, 2023

    The Abstract (p. 2) discusses human–AI co-creation, and Writing and composition (p. 12) describes collaborative writing as a possible use. The paper does not identify critical thinking, self-reflection, or data interpretation as the relevant areas. Practitioner Notes (p. 2) also cautions that even transparent AI use may breach an institution’s academic integrity policy, so collaboration is not necessarily policy-compliant. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Cites

Flagged first, then by citations

Research works printed but not linked: 6

  1. Curtis, G. J. (2022). Trends in plagiarism and cheating prevalence: 1990-2020 and beyond. In D. Rettinger & T. Bertram Gallant (Eds.), Cheating academic integrity (pp. 11-44). Jossey-Bass. https://www.wiley.com/enau/Cheating+Academic+Integrity%3A+Lessons+from+30+Years+of+Research-p9781119868170
  2. Dinneen, C. (2021). Students' use of digital translation and paraphrasing tools in written assignments on Direct Entry English Programs. English Australia Journal , 37 (1), 40-51. https://doi.org/10.3316/informit.748262877947586 · DOI
  3. Kumar, R., Mindzak, M., Eaton, S. E., & Morrison, R. (2022). AI & AI: Exploring the contemporary intersections of artificial intelligence and academic integrity . Canadian Society for the Study of Higher Education Annual Conference, Online. Werklund School of Education. https://tinyurl.com/ycknz8fd
  4. Kumar, R., Mindzak, M., & Racz, R. (2022). Who Wrote This? The Use of Artificial Intelligence in the Academy . http://hdl.handle.net/10464/16532
  5. Miller, A., & Young-Jones, A. D. (2012). Academic integrity: Online classes compared to face-toface classes. Journal of Instructional Psychology , 39 (3-4), Article 3-4. https://tinyurl.com/mrxnp9zy
  6. Wilder, N., Weßels, D., Gröpler, J., Klein, A., & Mundorf, M. (2021). Who is Responsible for Integrity in the Age of Artificial Intelligence? An Analysis Using the Example of Academic Writing. European Conference on Academic Integrity and Plagiarism 2021 , 179-181. https://tinyurl.com/2s4r5dyr

Cited by

Sorted by citations

Authors
Perkins, M.
Affiliations
British University Vietnam (VN)
Corresponding author
Perkins, Mike
Venue
Journal of University Teaching and Learning Practice
Published
2023-01-01
Volume
20
Issue
2
ISSN
1449-9789 (ISSN-L)
OpenAlex
W4321605350
Cited by
839
References
84
Citations by year
2026: 270 · 2025: 210 · 2024: 251 · 2023: 108
FWCI
209.33
Type
journal article
Language
en
Open access
Locations
  • Journal of University Teaching and Learning Practice (publishedVersion)
  • Research Online (University of Wollongong) (submittedVersion)
Topics
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
academic integrity, large language models, ChatGPT, generative AI in education, academic misconduct, plagiarism, automated writing evaluation, EFL learners, digital writing, composition teaching, human-AI co-creation, academic integrity policy, higher education, post-pandemic education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review