PreprintCited by 189Artificial Intelligence in EducationAcademic integrity and plagiarismArtificial Intelligence in Healthcare and Education

Better by You, better than Me? ChatGPT-3 as writing assistance in students’ essays.

Open access
Bašić, Željana, Banovac, Ana, Kružić, Ivana, Jerković, Ivan·2023·Humanities and Social Sciences Communications

Abstract

Aim: To compare students' essay writing performance with or without employing ChatGPT-3 as a writing assistant tool.Materials and methods: Eighteen students participated in the study (nine in control and nine in the experimental group that used ChatGPT-3). We scored essay elements with grades (A-D) and corresponding numerical values (4-1). We compared essay scores to students' GPTs, writing time, authenticity, and content similarity. Results: Average grade was C for both groups; for control (2.39 ± 0.71) and for experimental (2.00 ± 0.73). None of the predictors affected essay scores: group (P = 0.184), writing duration (P = 0.669), module (P = 0.388), and GPA (P = 0.532). The text unauthenticity was slightly higher in the experimental group (11.87% ±13.45 to 9.96% ± 9.81%), but the similarity among essays was generally low in the overall sample (the Jaccard similarity index ranging from 0 to 0.054). In the experimental group, AI classifier recognized more potential AI-generated texts. Conclusions: This study found no evidence that using GPT as a writing tool improves essay quality since the control group outperformed the experimental group in most parameters.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.35542/osf.io/n5m7s

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • ChatGPT has low performance in some topics, such as failing a parasitology exam for Korean medical students.

    “Additionally, other concerns include the current inability of ChatGPT to handle images as well as its low performance in some topics (e.g., failure to pass a parasitology exam for Korean medical students), and the issue of possible plagiarism [33,56,57,58,70,75].”

    From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [75]

    The Discussion (page unavailable) describes ChatGPT as lacking deep understanding of some topics and reports that students using it did not write higher-quality essays. The paper does not report a parasitology exam or a result involving Korean medical students, so it supports only the general concern about performance. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • ChatGPT use in health care education raises the issue of possible plagiarism.

    “Additionally, other concerns include the current inability of ChatGPT to handle images as well as its low performance in some topics (e.g., failure to pass a parasitology exam for Korean medical students), and the issue of possible plagiarism [33,56,57,58,70,75].”

    From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [75]

    The Abstract discusses plagiarism as a concern in ChatGPT-assisted education, and the Discussion reports slightly more plagiarism problems in the ChatGPT group. But the Materials and methods describe forensic-sciences students writing essays, not a study of health care education; the Discussion also says the essays showed no remarkable pairwise similarities. Page numbers are unavailable in our copy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • ChatGPT currently cannot handle images.

    “Additionally, other concerns include the current inability of ChatGPT to handle images as well as its low performance in some topics (e.g., failure to pass a parasitology exam for Korean medical students), and the issue of possible plagiarism [33,56,57,58,70,75].”

    From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [75]

    The Abstract describes ChatGPT-3.5’s text-based capabilities, and the Discussion addresses its use in essay writing; neither states whether it can handle images. Further searches for images, visual input, and picture processing found no passage addressing that capability. (LLM, conf=0.76)

    Experimental — accuracy not yet measured
  • In the UK, 88% of undergraduates reported using GenAI in assessments in 2024/25.

    “In the UK, an anonymous survey by the Higher Education Policy Institute 2 found that 88% of undergraduates reported using GenAI in assessments in 2024/25.”

    From Style, sentiment, and quality of undergraduate writing in the AI era: A cross-sectional and longitudinal analysis of 4,820 authentic empirical reports (2025) · cites this paper as [2]

    The Materials and methods section describes an experiment with 18 master’s students, and the Results report their essay grades, not a survey of UK undergraduates. Searches for the 88% figure, the 2024/25 period, and the Higher Education Policy Institute found no passage addressing the claimed survey result. (LLM, conf=0.78)

    Experimental — accuracy not yet measured
  • An anonymous survey of over 1000 British undergraduate students found that 30% reported using GenAI tools to generate text for coursework in November 2023, rising to 64% in December 2024.

    “Second, an anonymous survey 29 of over 1000 British undergraduate students revealed that 30% reported using GenAI tools to generate text for their coursework in November 2023, a figure that rose to 64% in December 2024 2 .”

    From Style, sentiment, and quality of undergraduate writing in the AI era: A cross-sectional and longitudinal analysis of 4,820 authentic empirical reports (2025) · cites this paper as [2]

    The Materials and methods section describes an essay-writing experiment with 18 master’s students, not a survey of British undergraduates. Searches for the November 2023 and December 2024 survey figures found no passage reporting the 30% or 64% figures. The paper therefore does not verify the trend attributed to it. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Pourhoseingholi MA, Hatamnejad MR, Solhpour A (2023) Does chatGPT (or any other artificial intelligence language tools) deserve to be included in authorship list? chatGPT and authorship. Gastroenterol Hepatol Bed Bench 16(1):435-437

Cited by

Sorted by citations

Authors
Bašić, Ž., Banovac, A., Kružić, I., & Jerković, I.
Affiliations
University of Split (HR)
Corresponding author
Banovac, Ana
Venue
Humanities and Social Sciences Communications
Published
2023-02-09
Volume
10
Issue
1
ISSN
2055-1045 (ISSN-L)
OpenAlex
W4388000629
Cited by
189
References
27
Citations by year
2026: 37 · 2025: 55 · 2024: 67 · 2023: 30
FWCI
44.36
Type
journal article
Language
en
Open access
Locations
  • Humanities and Social Sciences Communications (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • EdArXiv (OSF Preprints) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • EdArXiv (OSF Preprints) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
ChatGPT-3.5, AI-assisted writing, student essay writing, academic writing, AI-generated text detection, textual authenticity, essay quality, writing performance, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review