The DOI matches its Crossref record; 301 of the 303 references it cites were found in a registry or an index; 71 of the 197 sources its claims cite were checked for support.
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Open accessAbstract
The remarkable ability of large language models (LLMs) to comprehend, interpret, and generate complex language has rapidly integrated LLM-generated text into various aspects of daily life, where users increasingly accept it. However, the growing reliance on LLMs underscores the urgent need for effective detection mechanisms to identify LLM-generated text. Such mechanisms are critical to mitigating misuse and safeguarding domains like artistic expression and social networks from potential negative consequences. LLM-generated text detection, conceptualized as a binary classification task, seeks to determine whether an LLM produced a given text. Recent advances in this field stem from innovations in watermarking techniques, statistics-based detectors, and neural-based detectors. Human-assisted methods also play a crucial role. In this survey, we consolidate recent research breakthroughs in this field, emphasizing the urgent need to strengthen detector research. Additionally, we review existing datasets, highlighting their limitations and developmental requirements. Furthermore, we examine various LLM-generated text detection paradigms, shedding light on challenges like out-of-distribution problems, potential attacks, real-world data issues, and ineffective evaluation frameworks. Finally, we outline intriguing directions for future research in LLM-generated text detection to advance responsible artificial intelligence. This survey aims to provide a clear and comprehensive introduction for newcomers while offering seasoned researchers valuable updates in the field.1
Overview
CoChat Summary
TL;DR
This survey reviews LLM-generated text detection, organizing methods into watermarking, statistics-based, neural-based, and human-assisted approaches, and argues robust detectors are urgently needed. It synthesizes datasets, benchmarks, evaluation metrics, and open challenges, concluding with future research directions.
Key findings
- The survey’s systematic literature review screened work up to November 2023 and identified 83 pertinent pieces of literature; most relevant research appeared in 2023.
- Detection methods are categorized into watermarking (data-driven, model-driven, post-processing), statistics-based (linguistic features, white-box, black-box), neural-based (feature classifiers, pre-trained classifiers, LLMs as detectors), and human-assisted methods.
- It reports that current detectors face out-of-distribution problems (cross-domain, cross-lingual, cross-LLMs), attacks (paraphrase, adversarial, prompt, training threat models), real-world data issues (mixed/human-edited text, data ambiguity), model-size effects, and a lack of effective evaluation framework.
- It cites selected performance figures from reviewed work: Hamed and Wu’s bigram-based system identified 98 of 100 LLM-written academic papers; LLMDet achieved 98.54% classification accuracy; GHOSTBUSTER achieved an average F1 of 99.0; fine-tuned BERT-family models averaged about 95% in-domain accuracy.
Methods
- Design: a systematic literature review following Kitchenham’s SLR framework, guided by the research question of prevailing LLM-generated text detection methods and their main challenges.
- Data: searches across Google Scholar, arXiv, Scopus, Web of Science, IEEE Xplore, Springer Link, ACL Anthology, and ACM Digital Library using Boolean search terms; focused on publicly accessible articles before November 2023; after de-duplication and manual screening, 83 works were included.
- Analysis: synthesizes datasets and benchmarks, detector paradigms, evaluation metrics, important issues, and future directions; no primary experiments are conducted.
Limitations
- Stated: current datasets and benchmarks are limited in attack types, domains, tasks, LLMs, and languages, and suffer from temporal staleness; there is a lack of effective evaluation framework; no detection technique is acknowledged as infallible; data ambiguity is not formally addressed by existing studies.
- Stated: the paper argues prior reviews are insufficient in depth and comprehensiveness, but it does not state limitations of its own survey.
- Evident: as a survey, it presents no new empirical validation, and its coverage is bounded by the search terms, databases, and the November 2023 cutoff with 83 included works.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
50 Valid · 21 Not valid · 106 Not checked · 68 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“In the initial stages of generative modeling's emergence, when it had yet to pose significant societal challenges, the emphasis was on assessing the truth or falsity of the content in LLM-generated text, with less regard for its source (Schuster et al. 2020b).”
What the sources say
- Disputedcites Schuster et al. 2020b sourceExperimental — accuracy not yet measured
The Introduction (page not supplied) describes earlier approaches as focusing on stylistic differences and identifying whether text was machine-generated, then calls this paper’s veracity-based benchmark the first of its kind. That is contrary to the claim that the early emphasis was on truth or falsity rather than source. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“However, it is pertinent to note some limitations of the HC3 dataset, such as the lack of diversity in prompts used for data creation.”
What the sources say
- Could not verifycites Guo et al. 2023 sourceExperimental — accuracy not yet measured
Section 4.1 (Dataset; page unavailable) describes a PubMedQA-based dataset and its two sets of generation instructions. Section 6 (Limitations and Future Work; page unavailable) discusses limitations of AuthentiGPT, but neither section addresses HC3 or a lack of diversity in prompts used to create it. Searches for HC3 and prompt diversity found no passage supporting the claim. (LLM, conf=0.74)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“undermine technical expertise (Rodriguez et al. 2022a; Aliman and Kester 2021)”
What the sources say
- Partly supportedcites Rodriguez et al. 2022a sourceExperimental — accuracy not yet measured
The Abstract says convincing synthetic text can undermine genuine research, and the Introduction says synthetic technical texts could hinder the scientific process. Those passages support a threat to scientific work, but they do not specifically say that technical expertise itself is undermined. (LLM, conf=0.78)
- Not matchedcites Aliman and Kester 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“promote plagiarism (Lee et al. 2023a)”
What the sources say
- Partly supportedcites Lee et al. 2023a sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) identifies code plagiarism as a concern associated with advances in AI code generation. It lists code vulnerabilities separately, however, and does not say that those vulnerabilities promote plagiarism. (LLM, conf=0.76)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“The Human ChatGPT Comparison Corpus (HC3) (Guo et al. 2023) stands as one of the initial open-source efforts to compare ChatGPT-generated text with human-written text.”
What the sources say
- Could not verifycites Guo et al. 2023 sourceExperimental — accuracy not yet measured
The Dataset section describes a PubMedQA-based comparison of human-written and GPT-generated text, not HC3. Searches for “HC3,” “Human ChatGPT Comparison Corpus,” and the claimed early open-source comparison found no passage addressing HC3 or its status among early efforts. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
64 more claims · 20 with their cited sources checked
Create a free account to see all 245 claims and to verify the 16 references still open.
- Probably not a substantial claim. 119
- It may state two separate claims as one. 92
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Training · p. 17
“| Yelp Dataset (Asghar 2016) | 700k |”
§ 4.1 Training · p. 19
“It comprises 1.5 million review text samples.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 6. Evaluation Metrics · p. 38
“False Positive (FP) refers to the result of the positive category (LLM-generate text) incorrectly classified by the model.”
§ 6. Evaluation Metrics · p. 40
“The FPR refers to the proportion of all actual human-written samples that are incorrectly detected as LLM-generated samples.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Training · p. 17
“| Climate Change Dataset (Littman and Wrubel 2019) | 4m |”
§ 4.1 Training · p. 19
“This dataset houses the tweet IDs for 39.6 million tweets related to climate change.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Training · p. 17
“| TOEFL11 (Blanchard et al. 2013) | 12k |”
§ 4.1 Training · p. 18
“it encompasses 1.1k essay samples across 11 languages: Arabic, Chinese, French, German, Hindi, Italian, Japanese, Korean, Spanish, Telugu, and Turkish.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.2 Evaluation Benchmarks · p. 21
“| MULTITuDE (Macko et al. 2023) | train test | ~4k ~3k | ~40k ~26k”
§ 4.2 Evaluation Benchmarks · p. 22
“This dataset consists of 74k machine-generated texts and 7k human-written texts across 11 languages”
4 more findings
Create a free account to see every finding of this check.
Citations
Cites
Items 201–303 of 303
Becker, B. A., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., & Santos, E. A. (2022) · arXiv · doi:10.48550/arxiv.2212.01020 · Cited by 36
Littman, J., & Wrubel, L. (2019) · Harvard Dataverse · doi:10.7910/dvn/5qccuu · Cited by 35
ChatLog: Carefully Evaluating the Evolution of ChatGPT Across TimePreprint
Tu, S., Li, C., Yu, J., Wang, X., Hou, L., & Li, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.14106 · Cited by 34
Vasilatos, C., Alam, M., Rahwan, T., Zaki, Y., & Maniatakos, M. (2023) · arXiv · doi:10.48550/arxiv.2305.18226 · Cited by 34
Nguyen-Son, H.-Q., Dao, M.-S., & Zettsu, K. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1246 · Cited by 33
Testing of Detection Tools for AI-Generated Text
Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023) · arXiv · doi:10.48550/arxiv.2306.15666 · Cited by 32
ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models
Liu, Y., Zhang, Z., Zhang, W., Yue, S., Zhao, X., Cheng, X., Zhang, Y., & Hu, H. (2023) · arXiv · doi:10.48550/arxiv.2304.07666 · Cited by 32
Electronic document data hiding technique using inter-character space
Chotikakamthorn, N. (2002) · IEEE Asia Pacific Conference on Circuits and Systems (APCCAS) · doi:10.1109/apccas.1998.743799 · Cited by 31
Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs
Kim, Z. M., Lee, K., Zhu, P., Raheja, V., & Kang, D. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.298 · Cited by 31
Liu, S., Liu, X., Wang, Y., Cheng, Z., Li, C., Zhang, Z., Lan, Y., & Shen, C. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.103 · Cited by 31
Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text
Dhaini, M., Poelman, W., & Erdoğan, E. (2023) · Student Research Workshop .../Proceedings of the Student Research Workshop ... · doi:10.26615/issn.2603-2821.2023_001 · Cited by 31
Red Teaming Language Model Detectors with Language Models
Shi, Z., Wang, Y., Yin, F., Chen, X., Chang, K.-W., & Hsieh, C.-J. (2023) · arXiv · doi:10.48550/arxiv.2305.19713 · Cited by 30
Detection of ChatGPT fake science with the xFakeSci learning algorithm
Hamed, A. A., & Wu, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-66784-6 · Cited by 30
Harpy, a connected speech recognition system
Lowerre, B. T., & Reddy, B. R. (1976) · The Journal of the Acoustical Society of America · doi:10.1121/1.2003013 · Cited by 29
Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023) · arXiv · doi:10.48550/arxiv.2307.13528 · Cited by 29
MGTBench: Benchmarking Machine-Generated Text DetectionPreprint
He, X., Shen, X., Chen, Z., Michael Backes, A., & Zhang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.14822 · Cited by 29
Robustness to Modification with Shared Words in Paraphrase Identification
Shi, Z., & Huang, M. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.16 · Cited by 28
Bao, G., Zhao, Y., Teng, Z., Yang, L., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2310.05130 · Cited by 28
Computer-Generated Text Detection Using Machine Learning: A Systematic Review
Beresneva, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-41754-7_43 · Cited by 27
A Cognitive Regularizer for Language Modeling
Wei, J., Meister, C., & Cotterell, R. (2021) · doi:10.18653/v1/2021.acl-long.404 · Cited by 27
Chakraborty, M., Tonmoy, S. M. T. I., Zaman, S. M. M., Gautam, S., Kumar, T., Sharma, K., Barman, N. R., Gupta, C., Jain, V., Chadha, A. R., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.136 · Cited by 27
Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScorePreprint
Wu, J., Zhan, R., Wong, D. F., Yang, S., Liu, X., Chao, L. S., & Zhang, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.04286 · Cited by 26
Self-citationClassification of Human- and AI-Generated Texts for English, French, German, and Spanish
Schaaff, K., Schlippe, T., & Mindner, L. (2023) · arXiv · doi:10.48550/arxiv.2312.04882 · Cited by 26
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
Zeng, Z., Sha, L., Li, Y., Yang, K., Gašević, D., & Chen, G. (2023) · arXiv · doi:10.48550/arxiv.2307.12267 · Cited by 25
Beyond Black Box AI generated Plagiarism Detection: From Sentence to Document Level
Quidwai, A., Li, C., & Dube, P. (2023) · Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) · doi:10.18653/v1/2023.bea-1.58 · Cited by 24
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
Hans, A., Schwarzschild, A., Cherepanova, V., Kazemi, H., Saha, A., Goldblum, M., Geiping, J., & Goldstein, T. (2024) · arXiv · doi:10.48550/arxiv.2401.12070 · Cited by 24
DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text
Yang, X., Cheng, W., Wu, Y., Petzold, L., Wang, W. Y., & Chen, H. (2023) · arXiv · doi:10.48550/arxiv.2305.17359 · Cited by 23
Robust distortion-free watermarks for language models
Kuditipudi, R., Thickstun, J., Hashimoto, T., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2307.15593 · Cited by 23
Machine Translation Detection from Monolingual Web-Text
Arase, Y., & Zhou, M. (2013) · Cited by 23
Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated TextPreprint
Dhaini, M., Poelman, W., & Erdogan, E. (2023) · Recent Advances in Natural Language Processing · doi:10.48550/arxiv.2309.07689 · Cited by 21
GPT-who: An Information Density-based Machine-Generated Text Detector
Venkatraman, S., Uchendu, A., & Lee, D. (2023) · arXiv · doi:10.48550/arxiv.2310.06202 · Cited by 21
Large Language Models can be Guided to Evade AI-Generated Text Detection
Lu, N., Liu, S., He, R., Wang, Q., Ong, Y.-S., & Tang, K. (2023) · arXiv · doi:10.48550/arxiv.2305.10847 · Cited by 21
GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content
Chen, Y., Kang, H., Zhai, V., Li, L., Singh, R., & Raj, B. (2023) · arXiv · doi:10.48550/arxiv.2305.07969 · Cited by 21
J-Guard: Journalism Guided Adversarially Robust Detection of AI-generated NewsPreprint
Kumarage, T., Bhattacharjee, A., Padejski, D., Roschke, K., Gillmor, D., Ruston, S. W., Liu, H., & Garland, J. (2023) · International Joint Conference on Natural Language Processing · doi:10.48550/arxiv.2309.03164 · Cited by 20
Uniform Information Density Effects on Syntactic Choice in Hindi
Jain, A., Singh, V. K., Ranjan, S., Rajkumar, R. P., & Agarwal, S. (2018) · Cited by 20
TopFormer: Topology-Aware Authorship Attribution of Deepfake Texts with Diverse Writing Styles
Uchendu, A., Le, T., & Lee, D. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia240647 · Cited by 20
On pushing DeepFake Tweet Detection capabilities to the limits
Gambini, M., Fagni, T., Falchi, F., & Tesconi, M. (2022) · 14th ACM Web Science Conference 2022 · doi:10.1145/3501247.3531560 · Cited by 19
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection
Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Whitehouse, C., Afzal, O. M., Mahmoud, T., Sasaki, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.14902 · Cited by 19
Attacking Neural Text Detectors
Wolff, M., & Wolff, S. (2020) · arXiv · doi:10.48550/arxiv.2002.11768 · Cited by 18
Characterizing Social Spambots by their Human Traits
Giorgi, S., Ungar, L., & Schwartz, H. A. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.457 · Cited by 17
GPTs Don’t Keep Secrets: Searching for Backdoor Watermark Triggers in Autoregressive Language Models
Lucas, E., & Havens, T. C. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.21 · Cited by 17
Koike, R., Kaneko, M., & Okazaki, N. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.841 · Cited by 17
How effectively can machines defend against machine-generated fake news? An empirical study
Bhat, M. M., & Parthasarathy, S. (2020) · Proceedings of the First Workshop on Insights from Negative Results in NLP · doi:10.18653/v1/2020.insights-1.7 · Cited by 16
Ghostbuster: Detecting Text Ghostwritten by Large Language Models
Verma, V., Fleisig, E., Tomlin, N., & Klein, D. (2023) · arXiv · doi:10.48550/arxiv.2305.15047 · Cited by 16
Beat LLMs at their own game: Zero-shot LLM-generated text detection via querying ChatGPT
Zhu, B., Yuan, L., Cui, G., Chen, Y., Fu, C., He, B., Deng, Y., Liu, Z., Sun, M., & Gu, M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.463 · Cited by 16
Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection
Peng, X., Zhou, Y., He, B., Sun, L., & Sun, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.00412 · Cited by 16
Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text
Yang, L., Jiang, F., & Li, H. (2024) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000250 · Cited by 16
Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text
Yang, L., Jiang, F., & Li, H. (2023) · arXiv · doi:10.48550/arxiv.2307.11380 · Cited by 16
Liu, Z., Yao, Z., Li, F., & Luo, B. (2023) · arXiv · doi:10.48550/arxiv.2306.05524 · Cited by 16
Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection
Peng, X., Zhou, Y., He, B., Sun, L., & Sun, Y. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.644 · Cited by 16
A Statistical Turing Test for Generative ModelsPreprint
Helm, H. S., Priebe, C. E., & Yang, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.08913 · Cited by 15
Watermarking Pre-trained Language Models with Backdooring
Gu, C., Huang, C., Zheng, X., Chang, K.-W., & Hsieh, C.-J. (2022) · arXiv · doi:10.48550/arxiv.2210.07543 · Cited by 15
Mosca, E., Abdalla, M. H. I., Basso, P., Musumeci, M., & Groh, G. (2023) · Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023) · doi:10.18653/v1/2023.trustnlp-1.17 · Cited by 14
From Text to Source: Results in Detecting Large Language Model-Generated ContentPreprint
Antoun, W., Sagot, B., & Seddah, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.13322 · Cited by 14
Koike, R., Kaneko, M., & Okazaki, N. (2023) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2311.08369 · Cited by 14
Towards an understanding and explanation for mixed-initiative artificial scientific text detection
Weng, L., Liu, S., Zhu, H., Sun, J., Kam-Kwai, W., Han, D., Zhu, M., & Chen, W. (2024) · Information Visualization · doi:10.1177/14738716241240156 · Cited by 14
A deep fusion model for human $vs$. Machine-generated essay classification
Corizzo, R., & Leal-Arenas, S. (2023) · 2023 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191322 · Cited by 13
Raidar: geneRative AI Detection viA Rewriting
Mao, C., Vondrick, C., Wang, H., & Yang, J. (2024) · arXiv · doi:10.48550/arxiv.2401.12970 · Cited by 13
Assaying on the Robustness of Zero-Shot Machine-Generated Text DetectorsPreprint
Zhang, Y., Zhang, Z., Wang, L., Tan, T. N., & Jin, R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.12918 · Cited by 12
DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text DetectionPreprint
Yu, X., Qi, Y., Chen, K., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.12519 · Cited by 12
SciGen: a Dataset for Reasoning-Aware Text Generation from Scientific Tables
Moosavi, N. S., Rücklé, A., Roth, D., & Gurevych, I. (2021) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · Cited by 12
(2022) · doi:10.48550/arxiv.2210.08726 · Cited by 12
Hamed, A. A., & Wu, X. (2023) · Research Square · doi:10.21203/rs.3.rs-2851222/v1 · Cited by 11
How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Prompts
Kumarage, T., Sheth, P., Moraffah, R., Garland, J., & Liu, H. (2023) · arXiv · doi:10.48550/arxiv.2310.05095 · Cited by 11
MAGE: Machine-generated Text Detection in the Wild
Li, Y., Li, Q., Cui, L., Bi, W., Wang, Z., Wang, L., Yang, L., Shi, S., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.13242 · Cited by 10
Towards a Robust Detection of Language Model-Generated Text: Is ChatGPT that easy to detect?
Antoun, W., Mouilleron, V., Sagot, B., & Seddah, D. (2023) · arXiv · doi:10.48550/arxiv.2306.05871 · Cited by 9
Supervised machine-generated text detectors: Family and scale matters
Sarvazyan, A. M., González, J. Á., Rosso, P., & Franco-Salvador, M. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-42448-9_11 · Cited by 9
The Effectiveness of Free Software for Detecting AI-Generated Writing
Price, G., & Sakellarios, M. D. (2023) · International Journal of Teaching Learning and Education · doi:10.22161/ijtle.2.6.4 · Cited by 9
Mutation-Based Adversarial Attacks on Neural Text DetectorsPreprint
Liang, G., Guerrero, J., & Alsmadi, I. M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.05794 · Cited by 9
Origin Tracing and Detecting of LLMs
Li, L., Wang, P., Ren, K., Sun, T., & Qiu, X. (2023) · arXiv · doi:10.48550/arxiv.2304.14072 · Cited by 8
Zero-Shot Detection of LLM-Generated Text using Token Cohesiveness
Ma, S., & Wang, Q. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.971 · Cited by 8
Text Fluoroscopy: Detecting LLM-Generated Text through Intrinsic Features
Yu, X., Chen, K., Yang, Q., Zhang, W., & Yu, N. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.885 · Cited by 8
Desaire, H., Chua, A. E., Isom, M., Jarosova, R., & Hua, D. C. (2023) · arXiv.org · doi:10.48550/arxiv.2303.16352 · Cited by 7
Markowitz, D. M., Hancock, J. T., & Bailenson, J. N. (2023) · doi:10.31234/osf.io/mnyz8 · Cited by 6
HANSEN: Human and AI Spoken Text Benchmark for Authorship Analysis
Tripto, N. I., Uchendu, A., Le, T., Setzu, M., Giannotti, F., & Lee, D. (2023) · arXiv · doi:10.48550/arxiv.2310.16746 · Cited by 6
HANSEN: Human and AI Spoken Text Benchmark for Authorship Analysis
Tripto, N., Uchendu, A., Le, T., Setzu, M., Giannotti, F., & Lee, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.916 · Cited by 6
CogIE: An Information Extraction Toolkit for Bridging Texts and CogNet
Jin, Z., Chen, Y., Sui, D., Wang, C., Xue, Z., & Zhao, J. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.11 · Cited by 5
Long-form analogies generated by chatGPT lack human-like psycholinguistic properties
Seals, S. M., & Shalin, V. L. (2023) · arXiv · doi:10.48550/arxiv.2306.04537 · Cited by 5
Exploring Semantic Perturbations on GroverPreprint
Ji, Z., Kulkarni, P., Neskovic, M., Nolan, K., & Xu, Y. L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.00509 · Cited by 4
Who wrote this code? Watermarking for code generation
Lee, T., Hong, S., Ahn, J., Hong, I., Lee, H., Yun, S., Shin, J., & Kim, G. (2023) · arXiv · doi:10.48550/arxiv.2305.15060 · Cited by 4
AuthentiGPT: Detecting Machine-Generated Text via Black-Box Language Models Denoising
Guo, Z., & Yu, S. (2023) · arXiv · doi:10.48550/arxiv.2311.07700 · Cited by 4
Cowap, A., Graham, Y., & Foster, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.665 · Cited by 4
A Survey of Text Watermarking in the Era of Large Language ModelsPreprint
Liu, A., Pan, L., Lu, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.07913 · Cited by 4
Robust Multi-bit Natural Language Watermarking through Invariant FeaturesPreprint
Yoo, K., Ahn, W., Jang, J., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.01904 · Cited by 4
Tang, R., Feng, Q., Liu, N., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.11470 · Cited by 3
Munyer, T., Tanvir, A., Das, A., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.05773 · Cited by 3
ConDA: Contrastive Domain Adaptation for AI-generated Text Detection
Bhattacharjee, A., Kumarage, T., Moraffah, R., & Liu, H. (2023) · arXiv · doi:10.48550/arxiv.2309.03992 · Cited by 3
TOPFORMER: Topology-Aware Authorship Attribution of Deepfake Texts with Diverse Writing StylesPreprint
Uchendu, A., Lê, T. H., & Lee, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.12934 · Cited by 3
LLM-as-a-Coauthor: Can Mixed Human-Written and Machine-Generated Text Be Detected?
Zhang, Q., Gao, C., Chen, D., Huang, Y., Huang, Y., Sun, Z., Zhang, S., Li, W., Fu, Z., Wan, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.05952 · Cited by 3
SemStamp: A semantic watermark with paraphrastic robustness for text generation
Hou, A. B., Zhang, J., He, T., Wang, Y., Chuang, Y.-S., Wang, H., Shen, L., Van Durme, B., Khashabi, D., & Tsvetkov, Y. (2023) · arXiv · doi:10.48550/arxiv.2310.03991 · Cited by 3
Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey
Ghosal, S. S., Chakraborty, S., Geiping, J., Huang, F., Manocha, D., & Bedi, A. S. (2023) · arXiv · doi:10.48550/arxiv.2310.15264 · Cited by 3
Henrique, D. S. G., Kucharavy, A., & Guerraoui, R. (2023) · arXiv · doi:10.48550/arxiv.2304.08968 · Cited by 2
Detection of ChatGPT Fake Science with the xFakeSci Learning AlgorithmPreprint
Hamed, A. A., & Wu, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.11767 · Cited by 2
How do decoding algorithms distribute information in dialogue responses?
Venkatraman, S., He, H., & Reitter, D. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.70 · Cited by 2
Schneider, S., Steuber, F., Schneider, J. A. G., & Rodosek, G. D. (2023) · arXiv · doi:10.48550/arxiv.2310.16992 · Cited by 1
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Self-citationThe Age of Synthetic Realities: Challenges and Opportunities
Cardenuto, J. P., Yang, J., Padilha, R., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.11503
Ibrahim, H., Liu, F., Asim, R., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.13934
Self-Alignment with Instruction Backtranslation
Li, X., Yu, P., Zhou, C., Schick, T., Levy, O., Zettlemoyer, L., Weston, J., & Lewis, M. (2023) · arXiv · doi:10.48550/arxiv.2308.06259
Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text?
Bhattacharjee, A., & Liu, H. (2023) · arXiv · doi:10.48550/arxiv.2308.01284
Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts?
Uchendu, A., Lee, J., Shen, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.01002
Towards an Understanding and Explanation for Mixed-Initiative Artificial Scientific Text Detection
Weng, L., Zhu, M., Wong, K. K., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.05011
Tracing Text Provenance via Context-Aware Lexical Substitution
Yang, X., Zhang, J., Chen, K., Zhang, W., Ma, Z., Wang, F., & Yu, N. (2021) · arXiv · doi:10.48550/arxiv.2112.07873
Research works printed but not linked: 22
- Aich, 2022, Demystifying neural fake news via linguistic feature-based interpretation, 6586
- Christian, 2023, CNET secretly used AI on articles that didn’t disclose that fact, staff say, Futurism
- Cui, 2023, ChatLaw: Open-source legal large language model with integrated external knowledge bases, ArXiv preprint
- Deng, 2023, Efficient detection of LLM-generated texts with a Bayesian surrogate model, ArXiv preprint
- Dong, 2023, A survey for in-context learning, ArXiv preprint
- Giorgi, 2023, “I slept like a baby”: Using human traits to characterize deceptive ChatGPT and human text, 23
- Guo, 2020, Wiki-40B: Multilingual language model dataset, 2440
- He, 2023, Empirical analysis of beam search curse and search errors with model errors in neural machine translation, 91
- Kalinichenko, 2003, Digital libraries: Advanced methods and technologies, digital collections, D-Lib Magazine, 9, 1082
- Kitchenham, Barbara and StuartCharters. 2007. Guidelines for performing systematic literature reviews in software engineering. Technical Report, EBSE Technical Report EBSE-2007-01. pages 1–57. https://legacyfileshare.elsevier.com/promis_misc/525444systematicreviewsguide.pdf
- Kuditipudi, 2024, Robust distortion-free watermarks for language models, Transactions on Machine Learning Research
- Lambert, 2022, Illustrating reinforcement learning from human feedback (RLHF), HuggingFace Blog
- Liu, 2023, A private watermark for large language models, ArXiv preprint
- Liu, 2023, Check me if you can: Detecting ChatGPT-generated academic writing using CheckGPT, ArXiv preprint
- Ma, 2023, AI vs. human–differentiation analysis of scientific content generation, arXiv
- McCarthy, Philip M. 2005. An Assessment of the Range and Usefulness of Lexical Diversity Measures and the Potential of the Measure of Textual, Lexical Diversity (MTLD). Ph.D. thesis, The University of Memphis.
- Nicks, 2023, Language model detectors are easily optimized against
- Pagnoni, 2022, Threat scenarios and best practices to detect neural fake news, 1233
- Piccolo, 2023, Many bioinformatics programming tasks can be automated with ChatGPT, ArXiv preprint
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1, 9
- Yan, 2023, Detection of AI-generated essays in writing assessment, Psychological Testing and Assessment Modeling, 65, 125
- Yu, 2023, GPT paternity test: GPT generated text detection with GPT genetic inheritance, ArXiv preprint
Cited by
LLM Evaluators Recognize and Favor Their Own Generations
Panickssery, A., Bowman, S. R., & Feng, S. (2024) · arXiv · doi:10.48550/arxiv.2404.13076 · Cited by 559
Scalable watermarking for identifying large language model outputs
Dathathri, S., See, A., Ghaisas, S., Huang, P.-S., McAdam, R., Welbl, J., Bachani, V., Kaskasoli, A., Stanforth, R., Matejovicova, T., et al. (2024) · Nature · doi:10.1038/s41586-024-08025-4 · Cited by 132
Lyu, W., Wang, Y., Chung, T. (., Sun, Y., & Zhang, Y. (2024) · Proceedings of the Eleventh ACM Conference on Learning @ Scale · doi:10.1145/3657604.3662036 · Cited by 131
Mohammadabadi, S. M. S., Kara, B. C., Eyüpoğlu, C., Uzay, C., Tosun, M. S., & Karakuş, O. (2025) · Electronics · doi:10.3390/electronics14183580 · Cited by 89
Wu, Z., Dong, Y., Li, Y., & Shi, B. (2025) · European Journal of Operational Research · doi:10.1016/j.ejor.2025.04.032 · Cited by 42
Metadata
- Authors
- Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F.
- Affiliations
- University of Macau (MO)
- Peking University (CN)
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2025
- Volume
- 51
- Issue
- 1
- Pages
- 275-338
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W4406236959
- Cited by
- 212
- References
- 257
- Citations by year
- 2026: 106 · 2025: 91
- FWCI
- 79.02
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- United Mitochondrial Disease Foundation
- Universidade de Macau
- Fundo para o Desenvolvimento das Ciências e da Tecnologia (FDCT/060/2022/AFJ, FDCT/0070/2022/AMJ)
- State Administration for Science, Technology and Industry for National Defense
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- LLM-generated text detection, large language models, watermarking, detection dataset, out-of-distribution detection, adversarial attacks, evaluation framework, responsible AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-26, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-26 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.