The DOI matches its Crossref record; 301 of the 303 references it cites were found in a registry or an index; 71 of the 197 sources its claims cite were checked for support.
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Open accessAbstract
The remarkable ability of large language models (LLMs) to comprehend, interpret, and generate complex language has rapidly integrated LLM-generated text into various aspects of daily life, where users increasingly accept it. However, the growing reliance on LLMs underscores the urgent need for effective detection mechanisms to identify LLM-generated text. Such mechanisms are critical to mitigating misuse and safeguarding domains like artistic expression and social networks from potential negative consequences. LLM-generated text detection, conceptualized as a binary classification task, seeks to determine whether an LLM produced a given text. Recent advances in this field stem from innovations in watermarking techniques, statistics-based detectors, and neural-based detectors. Human-assisted methods also play a crucial role. In this survey, we consolidate recent research breakthroughs in this field, emphasizing the urgent need to strengthen detector research. Additionally, we review existing datasets, highlighting their limitations and developmental requirements. Furthermore, we examine various LLM-generated text detection paradigms, shedding light on challenges like out-of-distribution problems, potential attacks, real-world data issues, and ineffective evaluation frameworks. Finally, we outline intriguing directions for future research in LLM-generated text detection to advance responsible artificial intelligence. This survey aims to provide a clear and comprehensive introduction for newcomers while offering seasoned researchers valuable updates in the field.1
Overview
CoChat Summary
TL;DR
This survey reviews LLM-generated text detection, organizing methods into watermarking, statistics-based, neural-based, and human-assisted approaches, and argues robust detectors are urgently needed. It synthesizes datasets, benchmarks, evaluation metrics, and open challenges, concluding with future research directions.
Key findings
- The survey’s systematic literature review screened work up to November 2023 and identified 83 pertinent pieces of literature; most relevant research appeared in 2023.
- Detection methods are categorized into watermarking (data-driven, model-driven, post-processing), statistics-based (linguistic features, white-box, black-box), neural-based (feature classifiers, pre-trained classifiers, LLMs as detectors), and human-assisted methods.
- It reports that current detectors face out-of-distribution problems (cross-domain, cross-lingual, cross-LLMs), attacks (paraphrase, adversarial, prompt, training threat models), real-world data issues (mixed/human-edited text, data ambiguity), model-size effects, and a lack of effective evaluation framework.
- It cites selected performance figures from reviewed work: Hamed and Wu’s bigram-based system identified 98 of 100 LLM-written academic papers; LLMDet achieved 98.54% classification accuracy; GHOSTBUSTER achieved an average F1 of 99.0; fine-tuned BERT-family models averaged about 95% in-domain accuracy.
Methods
- Design: a systematic literature review following Kitchenham’s SLR framework, guided by the research question of prevailing LLM-generated text detection methods and their main challenges.
- Data: searches across Google Scholar, arXiv, Scopus, Web of Science, IEEE Xplore, Springer Link, ACL Anthology, and ACM Digital Library using Boolean search terms; focused on publicly accessible articles before November 2023; after de-duplication and manual screening, 83 works were included.
- Analysis: synthesizes datasets and benchmarks, detector paradigms, evaluation metrics, important issues, and future directions; no primary experiments are conducted.
Limitations
- Stated: current datasets and benchmarks are limited in attack types, domains, tasks, LLMs, and languages, and suffer from temporal staleness; there is a lack of effective evaluation framework; no detection technique is acknowledged as infallible; data ambiguity is not formally addressed by existing studies.
- Stated: the paper argues prior reviews are insufficient in depth and comprehensiveness, but it does not state limitations of its own survey.
- Evident: as a survey, it presents no new empirical validation, and its coverage is bounded by the search terms, databases, and the November 2023 cutoff with 83 included works.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
50 Valid · 21 Not valid · 106 Not checked · 68 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“In the initial stages of generative modeling's emergence, when it had yet to pose significant societal challenges, the emphasis was on assessing the truth or falsity of the content in LLM-generated text, with less regard for its source (Schuster et al. 2020b).”
What the sources say
- Disputedcites Schuster et al. 2020b sourceExperimental — accuracy not yet measured
The Introduction (page not supplied) describes earlier approaches as focusing on stylistic differences and identifying whether text was machine-generated, then calls this paper’s veracity-based benchmark the first of its kind. That is contrary to the claim that the early emphasis was on truth or falsity rather than source. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“However, it is pertinent to note some limitations of the HC3 dataset, such as the lack of diversity in prompts used for data creation.”
What the sources say
- Could not verifycites Guo et al. 2023 sourceExperimental — accuracy not yet measured
Section 4.1 (Dataset; page unavailable) describes a PubMedQA-based dataset and its two sets of generation instructions. Section 6 (Limitations and Future Work; page unavailable) discusses limitations of AuthentiGPT, but neither section addresses HC3 or a lack of diversity in prompts used to create it. Searches for HC3 and prompt diversity found no passage supporting the claim. (LLM, conf=0.74)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“undermine technical expertise (Rodriguez et al. 2022a; Aliman and Kester 2021)”
What the sources say
- Partly supportedcites Rodriguez et al. 2022a sourceExperimental — accuracy not yet measured
The Abstract says convincing synthetic text can undermine genuine research, and the Introduction says synthetic technical texts could hinder the scientific process. Those passages support a threat to scientific work, but they do not specifically say that technical expertise itself is undermined. (LLM, conf=0.78)
- Not matchedcites Aliman and Kester 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“promote plagiarism (Lee et al. 2023a)”
What the sources say
- Partly supportedcites Lee et al. 2023a sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) identifies code plagiarism as a concern associated with advances in AI code generation. It lists code vulnerabilities separately, however, and does not say that those vulnerabilities promote plagiarism. (LLM, conf=0.76)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“The Human ChatGPT Comparison Corpus (HC3) (Guo et al. 2023) stands as one of the initial open-source efforts to compare ChatGPT-generated text with human-written text.”
What the sources say
- Could not verifycites Guo et al. 2023 sourceExperimental — accuracy not yet measured
The Dataset section describes a PubMedQA-based comparison of human-written and GPT-generated text, not HC3. Searches for “HC3,” “Human ChatGPT Comparison Corpus,” and the claimed early open-source comparison found no passage addressing HC3 or its status among early efforts. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
64 more claims · 20 with their cited sources checked
Create a free account to see all 245 claims and to verify the 16 references still open.
- Probably not a substantial claim. 119
- It may state two separate claims as one. 92
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Training · p. 17
“| Yelp Dataset (Asghar 2016) | 700k |”
§ 4.1 Training · p. 19
“It comprises 1.5 million review text samples.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 6. Evaluation Metrics · p. 38
“False Positive (FP) refers to the result of the positive category (LLM-generate text) incorrectly classified by the model.”
§ 6. Evaluation Metrics · p. 40
“The FPR refers to the proportion of all actual human-written samples that are incorrectly detected as LLM-generated samples.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Training · p. 17
“| Climate Change Dataset (Littman and Wrubel 2019) | 4m |”
§ 4.1 Training · p. 19
“This dataset houses the tweet IDs for 39.6 million tweets related to climate change.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Training · p. 17
“| TOEFL11 (Blanchard et al. 2013) | 12k |”
§ 4.1 Training · p. 18
“it encompasses 1.1k essay samples across 11 languages: Arabic, Chinese, French, German, Hindi, Italian, Japanese, Korean, Spanish, Telugu, and Turkish.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.2 Evaluation Benchmarks · p. 21
“| MULTITuDE (Macko et al. 2023) | train test | ~4k ~3k | ~40k ~26k”
§ 4.2 Evaluation Benchmarks · p. 22
“This dataset consists of 74k machine-generated texts and 7k human-written texts across 11 languages”
4 more findings
Create a free account to see every finding of this check.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Self-citationGenerative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Research works printed but not linked: 22
- Aich, 2022, Demystifying neural fake news via linguistic feature-based interpretation, 6586
- Christian, 2023, CNET secretly used AI on articles that didn’t disclose that fact, staff say, Futurism
- Cui, 2023, ChatLaw: Open-source legal large language model with integrated external knowledge bases, ArXiv preprint
- Deng, 2023, Efficient detection of LLM-generated texts with a Bayesian surrogate model, ArXiv preprint
- Dong, 2023, A survey for in-context learning, ArXiv preprint
- Giorgi, 2023, “I slept like a baby”: Using human traits to characterize deceptive ChatGPT and human text, 23
- Guo, 2020, Wiki-40B: Multilingual language model dataset, 2440
- He, 2023, Empirical analysis of beam search curse and search errors with model errors in neural machine translation, 91
- Kalinichenko, 2003, Digital libraries: Advanced methods and technologies, digital collections, D-Lib Magazine, 9, 1082
- Kitchenham, Barbara and StuartCharters. 2007. Guidelines for performing systematic literature reviews in software engineering. Technical Report, EBSE Technical Report EBSE-2007-01. pages 1–57. https://legacyfileshare.elsevier.com/promis_misc/525444systematicreviewsguide.pdf
- Kuditipudi, 2024, Robust distortion-free watermarks for language models, Transactions on Machine Learning Research
- Lambert, 2022, Illustrating reinforcement learning from human feedback (RLHF), HuggingFace Blog
- Liu, 2023, A private watermark for large language models, ArXiv preprint
- Liu, 2023, Check me if you can: Detecting ChatGPT-generated academic writing using CheckGPT, ArXiv preprint
- Ma, 2023, AI vs. human–differentiation analysis of scientific content generation, arXiv
- McCarthy, Philip M. 2005. An Assessment of the Range and Usefulness of Lexical Diversity Measures and the Potential of the Measure of Textual, Lexical Diversity (MTLD). Ph.D. thesis, The University of Memphis.
- Nicks, 2023, Language model detectors are easily optimized against
- Pagnoni, 2022, Threat scenarios and best practices to detect neural fake news, 1233
- Piccolo, 2023, Many bioinformatics programming tasks can be automated with ChatGPT, ArXiv preprint
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1, 9
- Yan, 2023, Detection of AI-generated essays in writing assessment, Psychological Testing and Assessment Modeling, 65, 125
- Yu, 2023, GPT paternity test: GPT generated text detection with GPT genetic inheritance, ArXiv preprint
Cited by
LLM Evaluators Recognize and Favor Their Own Generations
Panickssery, A., Bowman, S. R., & Feng, S. (2024) · arXiv · doi:10.48550/arxiv.2404.13076 · Cited by 559
Scalable watermarking for identifying large language model outputs
Dathathri, S., See, A., Ghaisas, S., Huang, P.-S., McAdam, R., Welbl, J., Bachani, V., Kaskasoli, A., Stanforth, R., Matejovicova, T., et al. (2024) · Nature · doi:10.1038/s41586-024-08025-4 · Cited by 132
Lyu, W., Wang, Y., Chung, T. (., Sun, Y., & Zhang, Y. (2024) · Proceedings of the Eleventh ACM Conference on Learning @ Scale · doi:10.1145/3657604.3662036 · Cited by 131
Mohammadabadi, S. M. S., Kara, B. C., Eyüpoğlu, C., Uzay, C., Tosun, M. S., & Karakuş, O. (2025) · Electronics · doi:10.3390/electronics14183580 · Cited by 89
Wu, Z., Dong, Y., Li, Y., & Shi, B. (2025) · European Journal of Operational Research · doi:10.1016/j.ejor.2025.04.032 · Cited by 42
Metadata
- Authors
- Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F.
- Affiliations
- University of Macau (MO)
- Peking University (CN)
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2025
- Volume
- 51
- Issue
- 1
- Pages
- 275-338
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W4406236959
- Cited by
- 212
- References
- 257
- Citations by year
- 2026: 106 · 2025: 91
- FWCI
- 79.02
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- United Mitochondrial Disease Foundation
- Universidade de Macau
- Fundo para o Desenvolvimento das Ciências e da Tecnologia (FDCT/060/2022/AFJ, FDCT/0070/2022/AMJ)
- State Administration for Science, Technology and Industry for National Defense
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- LLM-generated text detection, large language models, watermarking, detection dataset, out-of-distribution detection, adversarial attacks, evaluation framework, responsible AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-26, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-26 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions. https://app.cochat.ai/papers/10.1162/coli_a_00549
- The paper
- Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025). A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions. Computational Linguistics. https://doi.org/10.1162/coli_a_00549
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.