Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection
Open accessAbstract
Large language models (LLMs) have exhibited remarkable capabilities in text generation tasks.However, the utilization of these models carries inherent risks, including but not limited to plagiarism, the dissemination of fake news, and issues in educational exercises.Although several detectors have been proposed to address these concerns, their effectiveness against adversarial perturbations, specifically in the context of student essay writing, remains largely unexplored.This paper aims to bridge this gap by constructing AIG-ASAP, an AI-generated student essay dataset, employing a range of text perturbation methods that are expected to generate high-quality essays while evading detection.Through empirical experiments, we assess the performance of current AIGC detectors on the AIG-ASAP dataset.The results reveal that the existing detectors can be easily circumvented using straightforward automatic adversarial attacks.Specifically, we explore word substitution and sentence substitution perturbation methods that effectively evade detection while maintaining the quality of the generated essays.This highlights the urgent need for more accurate and robust methods to detect AI-generated student essays in the education domain.Code and data are released for public use 1 .
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These findings are consistent with research showing that AI-generated texts can evade detection by subtly altering linguistic features.
“These findings are consistent with research showing that AI-generated texts can evade detec-tion by subtly altering linguistic features (e.g., Peng, et al., 2023).”
From A widely used Generative-AI detector yields zero false positives (2024) · cites this paper as Peng, et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441 · see citing works
GLTR: Statistical Detection and Visualization of Generated Text
Gehrmann, S., Strobelt, H., & Rush, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/p19-3019 · Cited by 470 · cited 2× · see citing works
“Current AI-generated text detection methods can be in general categorized into three categories (Lu et al., 2023): statistical (Gehrmann et al., 2019), watermarking (Kirchenbauer et al., 2023), and training-based (Gallé et al., 2021).”
BackgroundMethodChatGPT: The End of Online Exam Integrity?
Susnjak, T. (2022) · arXiv · doi:10.48550/arxiv.2212.09292 · Cited by 370 · cited 1× · see citing works
“There has been limited exploration of the effect of adversarial measures on detection accuracy (Susnjak, 2022; Liang et al., 2023), especially within the education domain.”
BackgroundA Watermark for Large Language Models
Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T. (2023) · arXiv · doi:10.48550/arxiv.2301.10226 · Cited by 194 · cited 2× · see citing works
“Current AI-generated text detection methods can be in general categorized into three categories (Lu et al., 2023): statistical (Gehrmann et al., 2019), watermarking (Kirchenbauer et al., 2023), and training-based (Gallé et al., 2021).”
BackgroundCan AI-Generated Text be Reliably Detected?
Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11156 · Cited by 161 · cited 5× · see citing works
“Recent research (Krishna et al., 2023; Sadasivan et al., 2023) demonstrate that simple paraphrasing of AI-generated text, regardless of whether it originates from the same model or different models, can remarkably evade the detector.”
BackgroundMethodInfluentialParaphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1195 · Cited by 150 · see citing works
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023) · arXiv · doi:10.48550/arxiv.2303.13408 · Cited by 150 · cited 3× · see citing works
“Krishna et al. (2023) train a generative model (DIPPER) to paraphrase paragraphs to evade detection.”
BackgroundAudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
Huang, R., Li, M., Yang, D., Shi, J., Chang, X., Ye, Z., Wu, Y., Hong, Z., Huang, J., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.12995 · Cited by 131 · see citing works
On the Possibilities of AI-Generated Text Detection
Chakraborty, S., Bedi, A. S., Zhu, S., An, B., Manocha, D., & Huang, F. (2023) · arXiv · doi:10.48550/arxiv.2304.04736 · Cited by 59 · see citing works
GPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · arXiv · doi:10.48550/arxiv.2304.02819 · Cited by 40 · see citing works
Towards a Robust Detection of Language Model-Generated Text: Is ChatGPT that easy to detect?
Antoun, W., Mouilleron, V., Sagot, B., & Seddah, D. (2023) · arXiv · doi:10.48550/arxiv.2306.05871 · Cited by 9 · see citing works
Flagged first, then by citations
Research works printed but not linked: 25
- Jonas Becker, Jan Philip Wahle, Terry Ruas, and Bela Gipp. 2023. Paraphrase detection: Human vs. ma- chine content. CoRR, abs/2303.13989. · DOI
- Moses Charikar. 2002. Similarity estimation techniques from rounding algorithms. In STOC, pages 380-388. ACM.
- Chen Chen, Jie Fu, and Lingjuan Lyu. 2023. A pathway towards responsible AI generated content. CoRR, abs/2303.01325. · DOI
- Evan Crothers, Nathalie Japkowicz, and Herna L. Viktor. 2022. Machine generated text: A comprehensive sur- vey of threat models and detection methods. CoRR, abs/2210.07321. · DOI · Paper page
- Tom Fawcett. 2006. An introduction to ROC analysis. Pattern Recognit. Lett., 27(8):861-874. · DOI · Paper page
- Matthias Gallé, Jos Rozen, Germán Kruszewski, and Hady Elsahar. 2021. Unsupervised and distribu- tional detection of machine-generated text. CoRR, abs/2111.02878. · arXiv
- Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. CoRR, abs/2301.07597. · DOI · Paper page
- Dirk Hovy. 2016. The enemy in your own camp: How well can we detect statistically-generated fake re- views -an adversarial study. In Proceedings of the 54th Annual Meeting of the Association for Compu- tational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 2: Short Papers. The Asso- ciation for Computer Linguistics. · DOI
- Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks V. S. Lakshmanan. 2020. Automatic detection of ma- chine generated text: A critical survey. In Proceed- ings of the 28th International Conference on Com- putational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pages 2296- 2309. International Committee on Computational Linguistics. · DOI · Paper page
- Cancan Jin, Ben He, Kai Hui, and Le Sun. 2018. TDNN: A two-stage deep neural network for prompt- independent automated essay scoring. In ACL (1), pages 1088-1097. Association for Computational Linguistics.
- Yongqi Li, Mayi Xu, Xin Miao, Shen Zhou, and Tieyun Qian. 2023. Large language models as counterfac- tual generator: Strengths and weaknesses. CoRR, abs/2305.14791. · DOI
- Yikang Liu, Ziyin Zhang, Wanyang Zhang, Shisen Yue, Xiaojing Zhao, Xinyuan Cheng, Yiwen Zhang, and Hai Hu. 2023a. Argugpt: evaluating, understanding and identifying argumentative essays generated by GPT models. CoRR, abs/2304.07666. · DOI · Paper page
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. · arXiv
- Zeyan Liu, Zijun Yao, Fengjun Li, and Bo Luo. 2023b. Check me if you can: Detecting chatgpt- generated academic writing using checkgpt. CoRR, abs/2306.05524. · arXiv
- Ning Lu, Shengcai Liu, Rui He, Qi Wang, and Ke Tang. 2023. Large language models can be guided to evade ai-generated text detection. CoRR, abs/2305.10847. · DOI · Paper page
- Fatemehsadat Mireshghallah, Justus Mattern, Sicun Gao, Reza Shokri, and Taylor Berg-Kirkpatrick. 2023. Smaller language models are better black- box machine-generated text detectors. CoRR, abs/2305.09859. · DOI
- Rabee Qasem, Banan Tantour, and Mohammed Maree. 2023. Towards the exploitation of llm-based chatbot for providing legal support to palestinian coopera- tives. CoRR, abs/2306.05827. · DOI
- Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. 2023. Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text. CoRR, abs/2306.05540. · DOI
- Hong Wang, Xuan Luo, Weizhi Wang, and Xifeng Yan. 2023a. Bot or human? detecting chatgpt imposters with A single question. CoRR, abs/2305.06424. · DOI
- Yongjie Wang, Chuang Wang, Ruobing Li, and Hui Lin. 2022. On the use of bert for automated essay scor- ing: Joint learning of multi-scale essay representation. In NAACL-HLT, pages 3416-3425. Association for Computational Linguistics.
- Zecong Wang, Jiaxi Cheng, Chen Cui, and Chenhao Yu. 2023b. Implementing BERT and fine-tuned roberta to detect AI generated news by chatgpt. CoRR, abs/2306.07401. · DOI
- Alex Wilson, Phil Blunsom, and Andrew D. Ker. 2014. Linguistic steganography on twitter: hierarchical lan- guage modeling with manual interaction. In Media Watermarking, Security, and Forensics 2014, San Francisco, CA, USA, February 2, 2014, Proceedings, volume 9028 of SPIE Proceedings, page 902803. SPIE. · DOI
- Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, and Hong Lin. 2023. Ai-generated content (AIGC): A survey. CoRR, abs/2304.06632. · DOI
- Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Informa- tion Processing Systems 2019, NeurIPS 2019, De- cember 8-14, 2019, Vancouver, BC, Canada, pages 9051-9062.
- Haolan Zhan, Xuanli He, Qiongkai Xu, Yuxiang Wu, and Pontus Stenetorp. 2023. G3detector: General gpt-generated text detector. CoRR, abs/2305.12680. · DOI
Cited by
Sorted by citations
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212 · see citing works
Pudasaini, S., Miralles-Pechuán, L., Lillis, D., & Llorens Salvador, M. (2025) · Journal of Academic Ethics · doi:10.1007/s10805-024-09576-x · Cited by 70 · see citing works
Enhancing the Robustness of AI-Generated Text Detectors: A Survey
Liu, X., Li, Y., & Li, K. (2025) · Mathematics · doi:10.3390/math13132145 · Cited by 10 · see citing works
Pudasaini, S., Miralles-Pechuán, L., Lillis, D., & Salvador, M. L. (2024) · arXiv · doi:10.48550/arxiv.2407.13105 · Cited by 8 · see citing works
“Similarly, paraphrasing, word substitution, and sentence substitution are used to evade binary classification training-based AIGC detection models [128] [129].”
BackgroundMethodNdacyayisenga, J. C. Y., Kidega, C., & Aciro Can, L. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09303-7 · Cited by 1 · see citing works
A widely used Generative-AI detector yields zero false positives
Gosling, S. D., Ybarra, K., & Angulo, S. K. (2024) · Aloma: Revista de Psicologia, Ciències de l'Educació i de l'Esport · doi:10.51698/aloma.2024.42.2.31-43 · Cited by 1 · see citing works
- Authors
- Peng, X., Zhou, Y., He, B., Sun, L., & Sun, Y.
- Affiliations
- Chinese Academy of Sciences (CN)
- University of Chinese Academy of Sciences (CN)
- Venue
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Published
- 2023
- Pages
- 10406-10419
- OpenAlex
- W4389520356
- Cited by
- 16
- References
- 12
- Citations by year
- 2026: 7 · 2025: 6 · 2024: 3
- FWCI
- 2.83
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62272439)
- Fundamental Research Funds for the Central Universities
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- AI-generated text detection, student essays, large language models, adversarial attacks, text perturbation, word substitution, academic integrity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
