PreprintCited by 3Artificial Intelligence in EducationAcademic Writing and PublishingAcademic integrity and plagiarism

Academic Writing with GPT-3.5: Reflections on Practices, Efficacy and Transparency

Open access
Buruk, Oğuz 'Oz'·2023

Abstract

The debate around the use of GPT-3.5 has been a popular topic among academics since the release of ChatGPT. Whilst some have argued for the advantages of GPT-3.5 in enhancing academic writing, others have raised concerns such as plagiarism, the spread of false information, and ecological issues. The need for finding ways to use GPT-3.5 models transparently has been voiced, and suggestions have been made on social media as to how to use GPT-3.5 models in a smart way. Nevertheless, to date, there is a lack of literature which clearly outlines how to use GPT-3.5 models in academic writing, how effective they are, and how to use them transparently. To address this, I conducted a personal experience experiment with GPT-3.5, specifically by using text-davinci-003 model of OpenAI, for writing this article. I identified five ways of using GPT-3.5: Chunk Stylist, Bullet-to-Paragraph, Talk Textualizer, Research Buddy, and Polisher. I reflected on their efficacy, and commented on their potential impact on writing ethics. Additionally, I provided a comprehensive document which shows the prompts I used, results I got from GPT-3.5, the final edits and visually compares those by showing the differences in percentage.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.31222/osf.io/vg5f6

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 12

  1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and others. 2020. Language models are few-shot learners. Advances in neural information processing systems 33: 1877-1901.
  2. Ke-Li Chiu, Annie Collins, and Rohan Alexander. 2021. Detecting hate speech with gpt-3. arXiv preprint arXiv:2103.12407. · arXiv
  3. Vanya Cohen and Aaron Gokaslan. 2020. OpenGPT-2: Open Language Models and Implications of Generated Text. XRDS 27, 1: 26- 30. https://doi.org/10.1145/3416063 · DOI
  4. Vijay Kumar Jain, Karthikeyan P Iyengar, and Raju Vaishya. 2022. Is the English language a barrier to the non-English-speaking authors in academic publishing? Postgraduate Medical Journal 98, 1157: 234-235. https://doi.org/10.1136/postgradmedj-2020- 139243 · DOI
  5. Tiffany H Kung, Morgan Cheatham, Arielle Medinilla, ChatGPT, Czarina Sillos, Lorie De Leon, Camille Elepano, Marie Madriaga, Rimel Aggabao, Giezel Diaz-Candido, and others. 2022. Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models. medRxiv: 2022-12.
  6. Kris McGuffie and Alex Newhouse. 2020. The radicalization risks of GPT-3 and advanced neural language models. arXiv preprint arXiv:2009.06807. · arXiv
  7. June Oshiro, Suzanne L. Caubet, Kelly E. Viola, and Jill M. Huber. 2020. Going Beyond "Not Enough Time": Barriers to Preparing Manuscripts for Academic Medical Journals. Teaching and Learning in Medicine 32, 1: 71-81. https://doi.org/10.1080/10401334.2019.165914 · DOI
  8. Tobias Schmohl, Alice Watanabe, Nadine Fröhlich, Dominikus Herzberg, and others. 2020. How Artificial Intelligence can improve the Academic Writing of Students. In Conference Proceedings. The Future of Education 2020.
  9. Chris Stokel-Walker and Richard Van Noorden. 2023. What ChatGPT and generative AI mean for science. Nature 614, 7947: 214-216. https://doi.org/10.1038/d41586-023-00340-6 · DOI · Paper page
  10. Angel Urbina-Garcia. 2020. What do we know about university academics' mental health? A systematic literature review. Stress and Health 36, 5: 563-585. https://doi.org/10.1002/smi.2956 · DOI
  11. 2020. Next chapter in artificial writing. Nature Machine Intelligence 2, 8: 419-419. https://doi.org/10.1038/s42256-020-0223-0 · DOI
  12. 2021. Compare text online -Show Text Difference - countwordsfree.com. Retrieved from https://countwordsfree.com/comparetexts 22. 2023. Tools such as ChatGPT threaten transparent science; here are our ground rules for their use. Nature 613, 7945: 612-612. https://doi.org/10.1038/d41586-023-00191-1 · DOI · Paper page

Cited by

Sorted by citations

Authors
Buruk, O. '.
Affiliations
Tampere University (FI)
Corresponding author
Buruk, Oğuz 'Oz'
Published
2023-03-06
OpenAlex
W4323363618
Cited by
3
References
2
Citations by year
2025: 1 · 2023: 2
Type
posted content
Language
en
Open access
Locations
  • MetArXiv (OSF Preprints) (submittedVersion)
  • MetArXiv (OSF Preprints) (submittedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
  • Social Sciences › Safety Research › Academic integrity and plagiarism
Keywords
GPT-3.5, academic writing, ChatGPT, prompt engineering, AI-assisted writing, ethics of writing, plagiarism, transparency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review