The DOI matches its DataCite record; 29 of the 31 references it cites were found in a registry or an index.
Automatic Detection of Generated Text is Easiest when Humans are Fooled
Open accessAbstract
Recent advancements in neural language modelling make it possible to rapidly generate vast amounts of human-sounding text. The capabilities of humans and automatic discriminators to detect machine-generated text have been a large source of research interest, but humans and machines rely on different cues to make their decisions. Here, we perform careful benchmarking and analysis of three popular sampling-based decoding strategies---top-$k$, nucleus sampling, and untruncated random sampling---and show that improvements in decoding methods have primarily optimized for fooling humans. This comes at the expense of introducing statistical abnormalities that make detection easy for automatic systems. We also show that though both human and automatic detector performance improve with longer excerpt length, even multi-sentence excerpts can fool expert human raters over 30% of the time. Our findings reveal the importance of using both human and automatic detectors to assess the humanness of text generation systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The rest of the zero-shot detectors (Solaiman et al., 2019; Gehrmann et al., 2019; Ippolito et al., 2019) also perform poorly after our attack.”
From Can AI-Generated Text be Reliably Detected? (2023) · cites this paper as Ippolito et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Solaiman et al. (2019) and Ippolito et al.(2020) also both present tools which showed an encouraging ability in detecting machine created text, although these tools were not tested against the latest generation of LLMs.”
From Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond (2023) · cites this paper as Ippolito et al.(2020)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Encoderonly Transformer models such as BERT (Devlin et al., 2019) have been shown to do very well at tasks requiring this understanding.”
BackgroundMethodInfluentialThe spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 2×
“It propogates quickly (Vosoughi et al., 2018), sets political agendas”
BackgroundSocial Media and Fake News in the 2016 Election
Allcott, H., & Gentzkow, M. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.211 · Cited by 6,767 · cited 2×
“It propogates quickly (Vosoughi et al., 2018), sets political agendas (Vargo et al., 2018), influences elections (Allcott and Gentzkow, 2017), and undermines user trust (Wang et al., 2012; Song et al., 2015).”
BackgroundGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865 · cited 2×
“textual word embeddings from a pre-trained BERT model without fine-tuning to compute a quality score for generated text (Zhang et al., 2020).”
MethodHierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1082 · Cited by 1,266 · cited 1×
“Top-k restricts the distribution to all but the k most likely tokens, where k is a constant (Fan et al., 2018).”
BackgroundFake It Till You Make It: Reputation, Competition, and Yelp Review Fraud
Luca, M., & Zervas, G. (2016) · Management Science · doi:10.1287/mnsc.2015.2304 · Cited by 1,263 · cited 1×
“(National Literacy Trust,2018). Aside from news, crowdturfing of fake reviews on websites such as Amazon and Yelp signifi- cantly undermines user trust (Wang et al.,2012; Song et al.,2015). According toLuca and Zervas (2016), approximately 16% of Yelp restaurant reviews were fraudulent as of 2012. Recently,Adelani et al…”
BackgroundAdversarial Learning for Neural Dialogue Generation
Li, J., Monroe, W., Shi, T., Jean, S., Ritter, A., & Jurafsky, D. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1230 · Cited by 955 · cited 2×
“Finally, discriminators for whether text is machine-generated are a promising research direction in adversarial training (Lin et al., 2017; Li et al., 2017) and in”
BackgroundGenerating Wikipedia by Summarizing Long SequencesPreprint
Liu, P. J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, Ł., & Shazeer, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.10198 · Cited by 784 · cited 2×
“, 2019), and Transformer-DMCA (Liu et al., 2018) are a few examples of large, publicly available models with this ability.”
Background
Research works printed but not linked: 4
- Timothy Niven and Hung-Yu Kao. 2019. Probing neu- ral network comprehension of natural language ar- guments. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguis- tics, pages 4658-4664, Florence, Italy. Association for Computational Linguistics.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715- 1725.
- Alan Turing. 1950. Computing machinery and intelligence-am turing. Mind, 59(236):433.
- Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. CoRR, abs/1905.12616. · arXiv
Cited by
Sorted by citations
Items 101–200 of 267
Artificially Generated Text Fragments Search in Academic Documents
Gritsai, G., Grabovoy, A., Kildyakov, A., & Chekhovich, Y. V. (2023) · Doklady Mathematics · doi:10.1134/s1064562423701211 · Cited by 14
How Large Language Models are Transforming Machine-Paraphrased PlagiarismPreprint
Wahle, J. P., Ruas, T., Kirstein, F., & Gipp, B. (2023) · doi:10.22541/au.167528149.95939897/v1 · Cited by 12
A deep fusion model for human $vs$. Machine-generated essay classification
Corizzo, R., & Leal-Arenas, S. (2023) · 2023 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191322 · Cited by 13
“…texts are also characterized by the repetitive use of words and expressions [24], and the use of the most frequently occurring words in a language, which can be gathered by utilizing automated detection methods that include the consideration of stop-words, unique words, and top lists [25].”
MethodClassifying human vs. AI text with machine learning and explainable transformer models
Masih, A., Afzal, B., Firdoos, S., Mahmood, J., Ali, A., Abdulnabi, M. S. H., & Balungu, D. M. (2025) · Scientific Reports · doi:10.1038/s41598-025-27377-z · Cited by 17
How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Prompts
Kumarage, T., Sheth, P., Moraffah, R., Garland, J., & Liu, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.94 · Cited by 27
Detecting multi-modal GAI-manipulated tourism review
Li, J., Zheng, W., & Guo, X. (2025) · Tourism Management · doi:10.1016/j.tourman.2025.105220 · Cited by 12
Through the Looking Glass: Learning to Attribute Synthetic Text Generated by Language Models
Munir, S., Batool, B., Shafiq, Z., Srinivasan, P., & Zaffar, F. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.155 · Cited by 11
“For scenarios where we need synthetic text generated using fine-tuned LMs, we limit text generation to the GPT2 LM. GPT2 has been shown to have state of the art performance in language generation tasks (Radford et al., 2019; Klein and Nabi, 2019).”
Wang, Q., & Li, H. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9030050 · Cited by 13
The Authenticity Gap in Human Evaluation
Ethayarajh, K., & Jurafsky, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.406 · Cited by 10
“The one area in which comparison-based evaluation still prevails is when conducting a Turing test—seeing whether annotators do better than chance when guessing whether a text is human-or machine-generated (Garbacea et al., 2019; Ippolito et al., 2020; Brown et al., 2020; Clark et al., 2021).”
MAGECODE: Machine-Generated Code Detection Method Using Large Language Models
Pham, H. N., Ha, H., Tong, V., Hoang, D., Tran, D., & Le, N. T. (2024) · IEEE Access · doi:10.1109/access.2024.3509987 · Cited by 11
“Consistent with findings on natural language content [14], [27], we also note that machine-generated code generally exhibits lower average values for both Rank and Log-Rank scores.”
LLM-Enhanced Multi-Task Joint Learning Model for Misinformation Detection
Ren, G., Li, J., Huang, T., Yang, Y., & Xie, R. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104305 · Cited by 9
Detecting AI-Generated Essays with Hybrid Supervised–Unsupervised NLP Techniques
Hole, S. R., Salotagi, S., Lakshmanan, M., Nuthakki, S., Nuthakki, Y., & Sagar, A. (2025) · International Conference on Artificial Intelligence and Signal Processing (AISP) · doi:10.1109/aisp68263.2025.11396327 · Cited by 9
STADEE: STAtistics-Based DEEp Detection of Machine Generated Text
Chen, Z., & Liu, H. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-4752-2_60 · Cited by 9
“Later, Ippolito et al. [19] found that setting 50 buckets of rank can achieve higher performance in their experiments.”
BackgroundSupervised machine-generated text detectors: Family and scale matters
Sarvazyan, A. M., González, J. Á., Rosso, P., & Franco-Salvador, M. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-42448-9_11 · Cited by 9
Lost in Transformation: Rediscovering LLM-Generated Campaigns in Social Media
Britta, G., Pohl, J., Winkelmann, H., Stampe, L., & Grimme, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47896-3_6 · Cited by 9
Authorship Verification in software engineering education
Rios, S., Zhang, Y., & de Oliveira, E. A. (2023) · ASCILITE Publications · doi:10.14742/apubs.2023.559 · Cited by 7
“Some approaches involve zero-shot detection which avoids the overhead of training models (Ippolito et al., 2019; Mitchell et al., 2023).”
BackgroundThe art of deception: humanizing AI to outsmart detection
Ayub, T., Malla, R. A., Khan, M. Y., & Ganaie, S. A. (2024) · Global Knowledge, Memory and Communication · doi:10.1108/gkmc-03-2024-0133 · Cited by 12
Covington, N. V., & Vruwink, O. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-024-00427-9 · Cited by 8
“…in a set, teaching assistants were asked to provide a humanlikeness rating on a scale from one to four, where 1 was “Definitely Human Written”; 2 was “Pos-sibly Human Written”; 3 was “Possibly Machine Generated”; and 4 was “Definitely Machine Generated” (Clark et al., 2021; Ippolito et al., 2020).”
A Comparison of DeepSeek and other LLMs
Gao, T., Jin, J., Ke, Z. T., & Moryoussef, G. (2026) · The American Statistician · doi:10.1080/00031305.2025.2611010 · Cited by 67
“Detection of AI-generated content is an emerging hot topic (Liu et al. 2019; Solaiman et al. 2019; Ippolito et al. 2020; Mitchell et al. 2023).”
Detecting Bot-Generated Text by Characterizing Linguistic Accommodation in Human-Bot Interactions
Bhatt, P., & Rios, A. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.286 · Cited by 8
“Automatically identifying bots online is well studied (Garcia-Silva et al., 2019; Herzig et al., 2019; Kosmajac and Keselj, 2019; Ippolito et al., 2020; Jawahar et al., 2020).”
BackgroundMethodCoAT: Corpus of artificial texts
Shamardina, T., Saidov, M., Fenogenova, A., Tumanov, A., Zemlyakova, A., Lebedeva, A., Gryaznova, E., Shavrina, T., Mikhailov, V., & Artemova, E. (2024) · Natural language processing. · doi:10.1017/nlp.2024.38 · Cited by 8
One-GPT: A One-Class Deep Fusion Model for Machine-Generated Text Detection
Corizzo, R., & Leal-Arenas, S. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386674 · Cited by 8
“The latter class frequently relies on the utilization of linguistic attributes, considering factors such as repetitiveness and emotional semantics [2] [13] [14] [15] [16] [17].”
BackgroundMARKMyWORDS: Analyzing and Evaluating Language Model Watermarks
Piet, J., Sitawarin, C., Fang, V., Mu, N., & Wagner, D. (2025) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00012 · Cited by 65
“Many schemes can be found in the literature, including OpenAI’s classifiers [24, 25], GLTR [26], DetectGPT [27], and others [28, 29, 30, 31, 32].”
Legal Language Modeling with Transformers
Peric, L., Mijic, S., Stammbach, D., & Ash, E. (2020) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000456079 · Cited by 7
“A complementary line of papers has explored automated detection of machine-generated text, mostly focusing on newspaper articles [6, 7].”
BackgroundResultIdentifying Automatically Generated Headlines using Transformers
Maronikolakis, A., Schütze, H., & Stevenson, M. (2021) · Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda (NLP4IF) · doi:10.18653/v1/2021.nlp4if-1.1 · Cited by 12
“Further, Ippolito et al. (2020) showed that shorter text is harder to detect, both for humans and machines.”
BackgroundDIALOG-22 RuATD Generated Text Detection
Lomonosov, M. S. U., Maloyan, N., Nutfullin, B., Lomonosov, M. S. U., Ilyshin, E., & Lomonosov, M. S. U. (2022) · Computational Linguistics and Intellectual Technologies · doi:10.28995/2075-7182-2022-21-394-401 · Cited by 12
“Either fine-tuning the proposed task or using probabilities distribution to make decision-based on them (Ippolito et al., 2019).”
MethodThe Effectiveness of Free Software for Detecting AI-Generated Writing
Price, G., & Sakellarios, M. D. (2023) · International Journal of Teaching Learning and Education · doi:10.22161/ijtle.2.6.4 · Cited by 9
Schneider, D. M., Mishra, A., Gluski, J. M., Shah, H., Ward, M., Brown, E. D. L., Sciubba, D. M., & Lo, S.-F. L. (2025) · Cureus · doi:10.7759/cureus.79086 · Cited by 12
“Ippolito et al. (2020) observed that perplexity-based detectors can achieve a sensitivity of approximately 80%, a specificity of 90%, and a false positive rate of 10% [18], though more recent investigations have suggested lower values [19].”
Ilic, P., & Carr, N. (2023) · 2023 IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie58773.2023.10343337 · Cited by 6
A Proposed Model for Distinguishing Between Human-Based and ChatGPT Content in Scientific Articles
Mohamed, T. A., Khafagy, M. H., Elsedawy, A. B., & Ismail, A. S. (2024) · IEEE Access · doi:10.1109/access.2024.3448315 · Cited by 8
“Additionally, Ippolito et al.’s work [4] demonstrated that the commonly used top-k sampling method in large language models (LLMs) tended to disproportionately favor high-likelihood words, resulting in statistical anomalies in the generated text that could be exploited for detection purposes.”
MethodHuynh, L. K., Nguyen, T., Goh, J., Kim, H., & Hong, J. B. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3481894 · Cited by 7
“Machine text detector: Machine-generated text is difficult to distinguish from human-written text [15], which has driven the need for machine text detectors such as RoBERTa (Section 3.”
BackgroundReinforcement learning for few-shot text generation adaptation
Cheng, P., Dai, J., Liu, J., Liu, J., & Jia, P. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126689 · Cited by 6
Automatic Detection for Machine-generated Texts is Easy
Lyu, M., Bao, C., Tang, J., Wang, T., & Liu, P. (2022) · IEEE Smart World Congress (SWC) · doi:10.1109/smartworld-uic-atc-scalcom-digitaltwin-pricomp-metaverse56740.2022.00223 · Cited by 5
“Ippolito trained a set of BERT-based binary classifiers to study generated text detection systems (cid:2) sensitivity to model structure, excerpt length and 3 decoding strategies applied to GPT-2, and analysed how human evaluators differ from automatic detectors in above aspects [6].”
BackgroundMethodOn the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook
Fan, M., Wang, C., Chen, C., Liu, Y., & Huang, J. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02375-w · Cited by 40
Stack More LLM’s: Efficient Detection of Machine-Generated Texts via Perplexity Approximation
Gritsai, G., Khabutdinov, I., & Grabovoy, A. (2024) · Doklady Mathematics · doi:10.1134/s1064562424602075 · Cited by 6
“Ippolito gave a great overview of all methods and their comparison [25].”
A Novel Benford’s Law-Driven Approach for Detecting Machine-Generated Text
Wang, Z., Zhang, C., & Ren, M. (2025) · ACM Transactions on Information Systems · doi:10.1145/3748305 · Cited by 6
“Main Ideas Criterion-Based Log-Likelihood [6] Token-wise log probability MGT often is higher Rank [5] Average token rank MGT often is higher Log-Rank [6] Log function to token rank MGT often is higher Entropy [5] Token-based entropy MGT often is higher GLTR [10] Token counts in the top n ranks MGT often shows higher v…”
InfluentialRojas-Simón, J., Ledeneva, Y. N., & García-Hernández, R. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-62836-8_31 · Cited by 5
Leveraging Multi-Model Linguistic Fusion for Enhanced AI Text Generation Evasion
Khan, A. A., & Rajagopalan, S. (2025) · International Generative AI and Computational Language Modelling Conference (GACLM) · doi:10.1109/gaclm67198.2025.11231821 · Cited by 18
Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System
Ippolito, D., Carlini, N., Lee, K., Nasr, M., & Yu, Y. W. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.28 · Cited by 7
Self-citationCowap, A., Graham, Y., & Foster, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.665 · Cited by 4
A Method for Distinguishing Model Generated Text and Human Written Text
Shimada, H., & Kimura, M. (2024) · Journal of Advances in Information Technology · doi:10.12720/jait.15.6.714-722 · Cited by 4
“Fundamentally, the discrimination of text generated by LLMs is primarily treated as a classification problem using statistical metrics as a representative approach [3].”
BackgroundMethodTowards AI-Generated Essay Classification Using Numerical Text Representation
Krawczyk, N., Probierz, B., & Kozak, J. (2024) · Applied Sciences · doi:10.3390/app14219795 · Cited by 4
Human Perception in Natural Language Generation
Mattei, L. D., Lai, H., Dell’Orletta⋄, F., & Nissim, M. (2021) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2021.gem-1.2 · Cited by 5
“Existing evaluations of GPT-2 models (Ippolito et al., 2020; De Mattei et al., 2020) have shown that while generated sentences were ranked lower in human perception than gold sentences, many gold sentences were also not perceived as human-like.”
BackgroundAdversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text
Cheng, Y., Sadasivan, V. S., Saberi, M., Saha, S., & Feizi, S. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-1588 · Cited by 3
Nassar, J. E., Farias, M. J., Singh, M., Dinh, P. V., Sahhar, M., Daher, M., Fallon, R. T., Marcaccio, S. E., Daniels, A. H., & Owens, B. D. (2025) · Orthopaedic Journal of Sports Medicine · doi:10.1177/23259671251371234 · Cited by 4
“12,24 Given these limitations, the traditional peer review process alone is insufficient to detect AI-generated content.”
Artificial text detection in Russian language: a BERT-based Approach
Posokhov, P., Skrylnikov, S., & Makhnytkina, O. V. (2022) · Computational Linguistics and Intellectual Technologies · doi:10.28995/2075-7182-2022-21-470-476 · Cited by 4
Exploring Naive Approaches to Tell Apart LLMs Productions from Human-written Text
Giudice, O., Maggi, A., & Nardelli, M. (2023) · International Conference on Natural Language Processing and Information Retrieval (NLPIR) · doi:10.1145/3639233.3639354 · Cited by 3
A Metric-Based Detection System for Large Language Model Texts
Le, L., & Tran, D. T. (2024) · ACM Transactions on Management Information Systems · doi:10.1145/3704739 · Cited by 4
“Traditionally, the typical works in detection of LLM texts train supervised detection models on top of deep embedding representations [2, 9, 17, 43].”
Beyond N-Grams: Enhancing String Kernels With Transformer-Guided Semantic Insights
Zaki, N., Alderei, R., Alketbi, M., Alkaabi, A., Alneyadi, F., & Zaki, N. (2025) · IEEE Access · doi:10.1109/access.2025.3576076 · Cited by 4
“Ippolito et al. [31] analyzed how different decoding strategies (top-k sampling, nucleus sampling) impact detection, revealing that while advancements in AI text generation optimize for fooling humans, they introduce statistical patterns that automated systems can exploit.”
Bilingual generated text detection through semantic and statistical analysis
Min, C., Zhang, R., & Liu, J. (2025) · Intelligent Data Analysis · doi:10.1177/1088467x241307192 · Cited by 3
“The statistical feature-based methods 9,12–14 exhibit certain limitations when dealing with high-quality LLM-generated text.”
An Ensemble-Based Approach for Generative Language Model Attribution
Abburi, H., Suesserman, M., Pudota, N., Veeramani, B., Bowen, E., & Bhattacharya, S. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-7254-8_54 · Cited by 3
Exploring the Profile of University Assessments Flagged as Containing AI-Generated Material
Gooch, D., Waugh, K., Richards, M., Slaymaker, M., & Woodthorpe, J. (2024) · ACM Inroads · doi:10.1145/3656478 · Cited by 3
“[10] noted that human discrimination of LLM outputs is generally lower than software detection, and that, to disguise their origins, LLMs tend to add statistical anomalies that potentially allow consistent detection of artificial solutions.”
BackgroundSEFD: Semantic-Enhanced Framework for Detecting LLM-Generated Text
He, W., Hou, B., Shang, T., Tarzanagh, D. A., Long, Q., & Shen, L. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825110 · Cited by 6
“The emergence of ChatGPT led to the development of zero-shot detectors [14], [16], [18]–[21], leveraging the statistical and topological properties of the LLM-generated text.”
TH-Bench: Evaluating Evading Attacks via Humanizing AI Text on Machine-Generated Text Detectors
Zheng, J., Wang, J., Sun, Z. T., Dong, W., Liu, Y., & He, X. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737418 · Cited by 6
“The model-based detectors [21, 15] modify LLMs like BERT [10] by adding a classification head and then fine-tuning the whole model on the MGT dataset.”
MIROSTAT: A NEURAL TEXT DECODING ALGORITHM THAT DIRECTLY CONTROLS PERPLEXITY
Basu, S., Ramachandran, G. S., Keskar, N. S., & Varshney, L. R. (2021) · International Conference on Learning Representations · Cited by 3
Fingerprinting Fine-tuned Language Models in the Wild
Diwan, N., Chakraborty, T., & Shafiq, Z. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.409 · Cited by 9
“Prior work has used the LM’s output word probability distribution to detect synthetic text (Ippolito et al., 2020; Gehrmann et al., 2019; Zellers et al., 2019).”
BackgroundMethodInfluentialFew-shot Controllable Style Transfer for Low-Resource Settings: A Study in Indian Languages.Preprint
Krishna, K., Nathani, D., García, X., Samanta, B., & Talukdar, P. P. (2021) · arXiv (Cornell University) · Cited by 3
Machine Learning Approaches to Identify AI-Generated Text: a Comparative Analysis
Mihir, T. K., Harsha, K. V. S., Nitya, S. Y., Krishna, G. B., Anamalamudi, S., & Sivarajan, S. (2024) · International Conference on Intelligent Computing and Emerging Communication Technologies (ICEC) · doi:10.1109/icec59683.2024.10837481 · Cited by 2
To Write or Not to Write as a Machine? That’s the Question
Sepúlveda-Torres, R., Martínez-Murillo, I., Saquete, E., Lloret, E., & Palomar, M. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3536938 · Cited by 2
Continual Origin Tracing of LLM-Generated Text
Li, H., & Wang, Q. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3729935 · Cited by 2
“Supervised classifiers are trained on labeled data using statistical features [22, 46] or neural representations [5, 6, 41, 52], typically by directly fine-tuning a pre-trained model such as BERT [11] or RoBERTa [29] to complete the classification task.”
Synergizing linguistic features and transformer networks for detecting AI-generated text
Yadagiri, A., Kumar, P., Poddar, Y., Pakray, P., & Chunka, C. (2026) · Knowledge and Information Systems · doi:10.1007/s10115-025-02637-6 · Cited by 2
Accurate Generated Text Detection Based on Deep Layer-wise Relevance Propagation
Guo, M., Liu, L., Guo, M., Liu, S., & Xu, Z. (2023) · IEEE International Conference on Big Data Analytics (ICBDA) · doi:10.1109/icbda57405.2023.10104941 · Cited by 3
“to detect generated texts [8], which included text content [9], text structure [10], generation technology [11] or used semantic coherence of texts to discover the generated texts [12].”
BackgroundCopyCAT: Masking Strategy Conscious Augmented Text for Machine Generated Text Detection
Liu, C., & Kao, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33374-3_29 · Cited by 2
Fine-Tuning GPT-3 for Synthetic Danish News Generation
Almasi, M., & Schiønning, A. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.4 · Cited by 2
“As human and machine detection methods presumably apply distinct techniques to spot synthetically generated text (Ippolito et al., 2020), a two dimensional evaluation provides a more nuanced insight into how GPT-3 performs on the task.”
BackgroundMethodInfluentialTextMachina: Seamless Generation of Machine-Generated Text Datasets
Sarvazyan, A. M., González, J. Á., & Franco-Salvador, M. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.462 · Cited by 4
Detection and Analysis of AI-Generated Malicious Content
Arafah, M. E., Al-Banna, A.-K., & Aladawi, A. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0832-6.ch011 · Cited by 3
On the Generalization and Adaptation Ability of Machine-Generated Text Detectors in Academic Writing
Liu, Y., Zhong, Z., Liao, Y., Sun, Z. T., Zheng, J., Wei, J., Gong, Q., Tong, F., Chen, Y., Zhang, Y., et al. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737408 · Cited by 7
“In contrast, model-based meth-ods [13, 9] typically integrate a classification head into a BERT model and fine-tune the augmented model on supervised datasets.”
ESPERANTO: Evaluating Synthesized Phrases to Enhance Robustness in AI Detection for Text Origination
Ayoobi, N., Knab, L., Cheng, W., Pantoja, D., Alikhani, H., Flamant, S., Kim, J., & Mukherjee, A. (2025) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3720553.3746665 · Cited by 9
“…AI text detection methods against our back-translation manipulation, we conduct experiments on nine detectors: six open-source models (RADAR [29], LLMDet [62], Likelihood [20], Rank [20, 30], Log-Rank, and ESAS [4]) and three commercial detectors (Pangram [14], GPTZero [22], and Ze-roGPT [65]).”
A Bayesian Social Platform for Inclusive and Evidence-Based Decision-Making
Devitt, S. K., Pearce, T. R., Chowdhury, A. K., & Mengersen, K. L. (2022) · Routledge eBooks · doi:10.4324/9780367808952-66 · Cited by 1
Sentence-level heuristic tree search for long text generation
Chen, Z., & Liu, Z. (2023) · Complex & Intelligent Systems · doi:10.1007/s40747-023-01244-8 · Cited by 2
MAUVE: Human-Machine Divergence Curves for Evaluating Open-Ended Text Generation.Preprint
Pillutla, K., Swayamdipta, S., Zellers, R., Thickstun, J., Choi, Y., & Harchaoui, Z. (2021) · arXiv (Cornell University) · Cited by 2
An Information Divergence Measure Between Neural Text and Human TextPreprint
Pillutla, K., Swayamdipta, S., Zellers, R., Thickstun, J., Welleck, S., Choi, Y., & Harchaoui, Z. (2021) · arXiv (Cornell University) · Cited by 2
Albert Gatt, A. (2025) · Oxford University Press eBooks · doi:10.1093/acrefore/9780199384655.013.896 · Cited by 2
Agarwal, A., & Uzair, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78498-9_5 · Cited by 1
Graded Suspiciousness of Adversarial Texts to Humans
Tonni, S. M., Faustini, P., & Dras, M. (2025) · Computational Linguistics · doi:10.1162/coli_a_00555 · Cited by 1
“…four widely used methods (Pruthi, Dhingra, and Lipton 2019; Alzantot et al. 2018; Jin et al. 2020; Garg and Ramakrishnan 2020), and analyze some characteristics of this dataset in light of observations of human accuracy in detecting machine-generated text (Ippolito et al. 2020; Clark et al. 2021).”
Distinguishing AI-Generated Text from Human-Written Text Using Machine Learning
Apte, S., Masnun, R., Lashiyal, A., & Kubek, M. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-90295-6_5 · Cited by 2
Master of Deceit: Comparative Analysis of Human and Machine-Generated Deceptive Text
Trinh, Q. M., Zarin, S., & Rezapour, R. (2025) · ACM Web Science Conference (WebSci) · doi:10.1145/3717867.3717914 · Cited by 1
Detecting AI-Generated Text: A Bi-GRU with Linguistic Features Approach
Hireche, A., Al-Dabet, S., Mediani, M., & Belkacem, A. N. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016416 · Cited by 2
“Their findings demonstrated that training detectors on adversarial examples enhances resilience to advanced AI outputs, underscoring the need for adaptive detection strategies [17].”
Detecting AI-Generated Vietnamese News Articles with Multilingual-E5 and BERT
Huynh, M.-P., Nguyen, H., Le, A.-C., & Truong, D. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-98170-8_11 · Cited by 1
Decoding AI-Generated Content Through Human Heuristics
Mitra, A., Mohanty, S., & Mishra, D. (2025) · Springer eBooks · doi:10.1007/978-3-031-77232-0_12 · Cited by 2
Deepfake Text Detection Using NLP and ML
Akashdeep, Mitra, H., & Gashroo, O. B. (2025) · International Conference on Computing, Intelligence, and Application (CIACON) · doi:10.1109/ciacon65473.2025.11189404 · Cited by 1
“[14 ].”
Using AI to control AI-assisted plagiarism in L2 writing: a critical review of the literature
Ibrahim, K. H. S. (2026) · Language Testing in Asia · doi:10.1186/s40468-026-00433-9 · Cited by 1
Detection of LLM-Generated Text vs. Human Text via DeepSeek-R1 Multi-Feature Fusion
Bu, X., Tang, M., Wang, J., Zhang, J., & Luo, P. H. (2026) · Information · doi:10.3390/info17040320 · Cited by 1
A Feature-based Linguistic and Statistical Framework for AI-Generated Text Detection
Sen, V., & Hasan, R. (2025) · International Conference on Machine Learning and Applications · doi:10.1109/icmla66185.2025.00030 · Cited by 1
“Manual detection of AGT by human reviewers has proven to be unreliable, especially as LLMs continue to improve in mimicking human-like style and reasoning [3].”
Loth, A. (2026) · ACM Web Science Conference (WebSci) · doi:10.1145/3795513.3807421 · Cited by 2
Adapting the Teaching of Academic Writing to the Emergence of Natural Language Generators
Ilieva, R. Y., & Tonchev, A. (2022) · International Conference on High Technology for Sustainable Development (HiTech) · doi:10.1109/hitech56937.2022.10145549 · Cited by 2
“Another weakness of GPT-3 and similar NLGs is that they are prone to inventing plausible facts, hence why they have been used to produce fake news [4].”
BackgroundCoRRPUS: Code-based Structured Prompting for Neurosymbolic Story Understanding
Dong, Y., Martin, L. J., & Callison-Burch, C. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.832 · Cited by 2
“…human-written stories, these models still struggle to generate coherent long-form text (See et al., 2019) & solve simple commonsense reasoning tasks (See Figure 1 for an example.) and therefore, end up only writing at human-level quality less than three-quarters of the time (Ippolito et al., 2020).”
BackgroundSelf-citationTowards Better Statistical Understanding of Watermarking LLMs
Cai, Z., Liu, S., Wang, H., Zhong, H., & Li, X. (2026) · Journal of the American Statistical Association · doi:10.1080/01621459.2026.2618290 · Cited by 17
“…by another LLM, the statistical properties of machine-generated texts deviate from the human-generated ones in aspects such as the distributions of token log-likelihoods (Gehrmann et al., 2019; Ippolito et al., 2019; Zellers et al., 2019; Solaiman et al., 2019; Tian, 2023; Mitchell et al., 2023).”
Distinguishing Chatbot from Human
Godghase, G. A., Agrawal, R., Obili, T., & Stamp, M. (2025) · Springer eBooks · doi:10.1007/978-3-031-83157-7_19 · Cited by 2
Detecting Auto-generated Texts with Language Model and Attacking the Detector
Saint Petersburg, R., & Orzhenovskii, M. (2022) · Computational Linguistics and Intellectual Technologies · doi:10.28995/2075-7182-2022-21-412-419 · Cited by 1
(2021) · PeerJ Computer Science · doi:10.7287/peerj-cs.443v0.1/reviews/2 · Cited by 1
Sadiq, S., & Ullah, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4494619 · Cited by 1
Estupinan-Villegas, G. (2024) · doi:10.31235/osf.io/de75c · Cited by 1
Perceptual Models of Machine-Edited Text
Merkhofer, E., Mendoza, M.-A., Marvin, R., & Henderson, J. J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.342
“In fact, the generation setting that best fools humans produces output that is easy to detect automatically (Ippolito et al., 2020).”
BackgroundOn factuality in neural language models
Nadeem, M. (2021)
Text Generation and Evaluation for Human-Machine Collaborative Writing
Clark, E. (2021) · ResearchWorks at the University of Washington (University of Washington)
Giới thiệu về DetectGPT – Công cụ phát hiện văn bản do máy tính tạo ra
Nam, L. H. (2023) · Tạp chí điện tử Khoa học và Công nghệ Giao thông · doi:10.58845/jstt.utt.2023.vn.3.45-54
Dankova, B. (2023) · Reinvention an International Journal of Undergraduate Research · doi:10.31273/reinvention.v16i1.964
Understanding the impact of fake quote retweets on genuine Twitter users using sentiment analysis
Shaikh, N., & Prasad, S. K. (2023) · AIP conference proceedings · doi:10.1063/5.0168539
Metadata
- Authors
- Ippolito, D., Duckworth, D., Callison-Burch, C., & Eck, D.
- Affiliations
- University of Pennsylvania (US)
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- Pages
- 1808-1822
- ISSN
- 0736-587X (ISSN-L)
- arXiv
- 1911.00650
- OpenAlex
- W3021153741
- Cited by
- 264
- References
- 31
- Citations by year
- 2026: 39 · 2025: 64 · 2024: 38 · 2023: 63 · 2022: 18 · 2021: 29 · 2020: 13
- FWCI
- 11.75
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency (FA8750-19-2-1004, FA8750)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- AI-generated text detection, decoding strategies, top-k sampling, nucleus sampling, random sampling, human raters, text generation evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
