CoChat
Sign Up
PreprintCited by 42Authorship Attribution and ProfilingDigital Media Forensic DetectionMisinformation and Its Impacts

To ChatGPT, or not to ChatGPT: That is the question!

Open access
Pegoraro, Alessandro, Kumari, Kavita, Fereidooni, Hossein, Sadeghi, Ahmad-Reza·2023·arXiv
DOI 10.48550/arxiv.2304.01487arXiv 2304.01487
Open this paper in CoChat — free accountOpen PDF

Abstract

ChatGPT has become a global sensation. As ChatGPT and other Large Language Models (LLMs) emerge, concerns of misusing them in various ways increase, such as disseminating fake news, plagiarism, manipulating public opinion, cheating, and fraud. Hence, distinguishing AI-generated from human-generated becomes increasingly essential. Researchers have proposed various detection methodologies, ranging from basic binary classifiers to more complex deep-learning models. Some detection techniques rely on statistical characteristics or syntactic patterns, while others incorporate semantic or contextual information to improve accuracy. The primary objective of this study is to provide a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. Additionally, we evaluated other AI-generated text detection tools that do not specifically claim to detect ChatGPT-generated content to assess their performance in detecting ChatGPT-generated content. For our evaluation, we have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains and user-generated responses from popular social networking platforms. The dataset serves as a reference to assess the performance of various techniques in detecting ChatGPT-generated content. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2304.01487

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • At present, AI-generated content cannot be reliably detected, especially if it is adapted from AI output.

    “the nature of Generative AI was such that it would be almost impossible to reliably detect AI-generated writing.”

    From Generative Artificial Intelligence: Implications and Considerations for Higher Education Practice (2023) · cites this paper as [15]

    The Abstract and 'C. Evaluated Tools and Algorithms' report that the evaluated methods could not consistently detect ChatGPT-generated text, supporting the difficulty of reliable detection at the time. The paper does not establish that AI-generated writing in general is inherently almost impossible to detect, or that text adapted from AI output is especially hard to detect. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 22

  1. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023, 2023. · arXiv
  2. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human prefer- ences. Advances in neural information processing systems, 30, 2017.
  3. Debby RE Cotton, Peter A Cotton, and J Reuben Shipway. Chatting and cheating: Ensuring academic integrity in the era of chatgpt. Innovations in Education and Teaching International, pages 1-12, 2023.
  4. Luigi De Angelis, Francesco Baglivo, Guglielmo Arzilli, Gaetano Pier- paolo Privitera, Paolo Ferragina, Alberto Eugenio Tozzi, and Caterina Rizzo. Chatgpt and the rise of large language models: The new ai-driven infodemic threat in public health. Available at SSRN 4352931, 2023.
  5. Michael Dowling and Brian Lucey. Chatgpt for (finance) research: The bananarama conjecture. Finance Research Letters, page 103662, 2023.
  6. Leon Fröhling and Arkaitz Zubiaga. Feature-based detection of automated language models: tackling gpt-2, gpt-3 and grover. PeerJ Computer Science, 7:e443, April 2021. https://peerj.com/articles/cs-443/#supplementary-material.
  7. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M Rush. Gltr: Statistical detection and visualization of generated text. arXiv preprint arXiv:1906.04043, 2019. http://gltr.io/. · arXiv
  8. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. How close is chatgpt to human experts? comparison corpus, evalu- ation, and detection. arXiv preprint arXiv:2301.07597, 2023. https://github.com/Hello-SimpleAI/chatgpt-comparison-detection. · arXiv
  9. Mohammad Khalil and Erkan Er. Will chatgpt get you caught? rethinking of plagiarism detection. arXiv preprint arXiv:2302.04335, 2023. · arXiv
  10. Tharindu Kumarage, Joshua Garland, Amrita Bhat- tacharjee, Kirill Trapeznikov, Scott Ruston, and Huan Liu. Stylometric detection of ai-generated text in twit- ter timelines. arXiv preprint arXiv:2303.03697, 2023. https://github.com/TSKumarage/Stylo-Det-AI-Gen-Twitter-Timelines. · arXiv
  11. Laida Kushnareva, Daniil Cherniavskii, Vladislav Mikhailov, Ekate- rina Artemova, Serguei Barannikov, Alexander Bernstein, Irina Pio- ntkovskaya, Dmitri Piontkovski, and Evgeny Burnaev. Artificial text detection via examining the topology of attention maps. arXiv preprint arXiv:2109.04825, 2021. https://github.com/danchern97/tda4atd. · arXiv
  12. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019. · arXiv
  13. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine- generated text detection using probability curvature. arXiv preprint arXiv:2301.11305, 2023. https://github.com/eric-mitchell/detect-gpt https://detectgpt.ericmitchell.ai/. · arXiv
  14. Sandra Mitrović, Davide Andreoletti, and Omran Ayoub. Chatgpt or human? detect and explain. explaining decisions of machine learn- ing model for detecting short chatgpt-generated text. arXiv preprint arXiv:2301.13852, 2023. · arXiv
  15. Reham Omar, Omij Mangukiya, Panos Kalnis, and Essam Mansour. Chatgpt versus traditional question answering for knowledge graphs: Current status and future directions towards knowledge graph chatbots. arXiv preprint arXiv:2302.06466, 2023. · arXiv
  16. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  17. Jürgen Rudolph, Samson Tan, and Shannon Tan. Chatgpt: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching, 6(1), 2023.
  18. Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203, 2019. https://openai-openai-detector.hf.space/ https://github.com/openai/gpt-2-output-dataset/tree/master/detector. · arXiv
  19. Teo Susnjak. Applying bert and chatgpt for sentiment analysis of lyme disease in scientific literature. arXiv preprint arXiv:2302.06474, 2023. · arXiv
  20. Ruixiang Tang, Yu-Neng Chuang, and Xia Hu. The science of detecting llm-generated texts. arXiv preprint arXiv:2303.07205, 2023. · arXiv
  21. Yee Hui Yeo, Jamil S Samaan, Wee Han Ng, Peng-Sheng Ting, Hirsh Trivedi, Aarshi Vipani, Walid Ayoub, Ju Dong Yang, Omer Liran, Brennan Spiegel, et al. Assessing the performance of chatgpt in answering questions regarding cirrhosis and hepatocellular carcinoma. medRxiv, pages 2023-02, 2023.
  22. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. Advances in neural information processing systems, 32, 2019. https://rowanzellers.com/grover/.

Cited by

Sorted by citations

Related

Authors
Pegoraro, A., Kumari, K., Fereidooni, H., & Sadeghi, A.-R.
Affiliations
Technische Universität Darmstadt (DE)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2304.01487
arXiv
2304.01487
OpenAlex
W4362678982
Cited by
39
References
0
Citations by year
2026: 2 · 2025: 4 · 2024: 18 · 2023: 14 · 2021: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
AI-generated text detection, large language models, benchmark dataset, detection method evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review