Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 102

The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.

Detecting AI-generated essays: the ChatGPT challenge

Cingillioglu, Ilker · 2023 · The International Journal of Information and Learning Technology

Abstract

Purpose: With the advent of ChatGPT, a sophisticated generative artificial intelligence (AI) tool, maintaining academic integrity in all educational settings has recently become a challenge for educators. This paper discusses a method and necessary strategies to confront this challenge. Design/methodology/approach: In this study, a language model was defined to achieve high accuracy in distinguishing ChatGPT-generated essays from human written essays with a particular focus on “not falsely” classifying genuinely human-written essays as AI-generated (Negative). Findings: Via support vector machine (SVM) algorithm 100% accuracy was recorded for identifying human generated essays. The author discussed the key use of Recall and F2 score for measuring classification performance and the importance of eliminating False Negatives and making sure that no actual human generated essays are incorrectly classified as AI generated. The results of the proposed model's classification algorithms were compared to those of AI-generated text detection software developed by OpenAI, GPTZero and Copyleaks. Practical implications: AI-generated essays submitted by students can be detected by teachers and educational designers using the proposed language model and machine learning (ML) classifier at a high accuracy. Human (student)-generated essays can and must be correctly identified with 100% accuracy even if the overall classification accuracy performance is slightly reduced. Originality/value: This is the first and only study that used an n-gram bag-of-words (BOWs) discrepancy language model as input for a classifier to make such prediction and compared the classification results of other AI-generated text detection software in an empirical way.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1108/ijilt-03-2023-0043

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 4 references

Research works printed but not linked: 15

  1. 2023, The Story of ChatGPT and OpenAI: the Evolution of GPT models, Medium
  2. 2023, ‘So what if ChatGPT wrote it?’ Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy, International Journal of Information Management, 71
  3. 1998, Profile hidden Markov models, Bioinformatics (Oxford), 14, 755
  4. 2022, How to cheat on your final paper: assigning AI for student writing, AI and Society, 1, 10.1007/s00146-022-01397-z · DOI · Paper page
  5. 2022, How to Spot AI-Generated Text
  6. 2013, Training and analysing deep recurrent neural networks, Part of Advances in Neural Information Processing Systems 26 (NIPS 2013)
  7. 1997, Hidden Markov models in speech and language processing, Corpus-based Methods in Language and Speech Processing, 27
  8. 2012, Advanced applications of neural networks and artificial intelligence: a review, International Journal of Information Technology and Computer Science, 4, 57, 10.5815/ijitcs.2012.06.08 · DOI · Paper page
  9. 1992, Robust part-of-speech tagging using a hidden Markov model, Computer Speech & Language, 6, 225, 10.1016/0885-2308(92)90019-Z · DOI · Paper page
  10. 2014, No One's Talking about the Amazing Chatbot that Passed the Turing Test 3 Years Ago
  11. 1985, Discourse strategies for generating natural-language text, Artificial Intelligence, 27, 1, 10.1016/0004-3702(85)90082-7 · DOI · Paper page
  12. 2022, A Thesis that Writes Itself: On the Threat of AI-Generated Essays within Academia
  13. 2023, ChatGPT Parent Open AI Gets ‘Game Changing’ Multibillion-Dollar Boost from Microsoft
  14. 2022, Automated essay writing: an AIED opinion, International Journal of Artificial Intelligence in Education, 32, 1119
  15. 1970, What does it mean to understand language?, Cognitive Science, 4, 209

Cited by

See all 102

Metadata

Authors
Cingillioglu, I.
Venue
The International Journal of Information and Learning Technology
Publisher
Emerald
Published
2023-05-01
Volume
40
Issue
3
Pages
259-268
ISSN
2056-4880
Cited by
102
References
4
Language
en
Licence
www.emerald.com/insight/site-policies (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Detecting AI-generated essays: the ChatGPT challenge. https://app.cochat.ai/papers/10.1108/ijilt-03-2023-0043
The paper
Cingillioglu, I. (2023). Detecting AI-generated essays: the ChatGPT challenge. The International Journal of Information and Learning Technology. https://doi.org/10.1108/ijilt-03-2023-0043

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.