DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
Open accessAbstract
The increasing fluency and widespread usage of large language models (LLMs) highlight the desirability of corresponding tools aiding detection of LLM-generated text. In this paper, we identify a property of the structure of an LLM's probability function that is useful for such detection. Specifically, we demonstrate that text sampled from an LLM tends to occupy negative curvature regions of the model's log probability function. Leveraging this observation, we then define a new curvature-based criterion for judging if a passage is generated from a given LLM. This approach, which we call DetectGPT, does not require training a separate classifier, collecting a dataset of real or generated passages, or explicitly watermarking generated text. It uses only log probabilities computed by the model of interest and random perturbations of the passage from another generic pre-trained language model (e.g., T5). We find DetectGPT is more discriminative than existing zero-shot methods for model sample detection, notably improving detection of fake news articles generated by 20B parameter GPT-NeoX from 0.81 AUROC for the strongest zero-shot baseline to 0.95 AUROC for DetectGPT. See https://ericmitchell.ai/detectgpt for code, data, and other project information.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
After paraphrasing, the AUROC of zero-shot detectors drops from 96.5% to 25.2%.
“After paraphrasing, the area under the receiver operating characteristic (AUROC) curves of zero-shot detectors (Mitchell et al., 2023) drops from 96 . 5% to 25 . 2%.”
From Can AI-Generated Text be Reliably Detected? (2023) · cites this paper as Mitchell et al., 2023
Section 5.2 reports that detection performance generally declines as machine-generated text is more heavily revised, supporting the direction of the claim. It does not report a drop from 96.5% to 25.2%; instead, it says DetectGPT's AUROC remains above 0.8 when nearly a quarter of the text is replaced. (LLM, conf=0.88)
Experimental — accuracy not yet measuredIn many detection methods, perplexity is a crucial component.
“In particular, the detector LLM is not given explicit access to information such as perplexity, which in many detection methods is a crucial component (Mitchell et al., 2023;Hans et al., 2024).”
From LLM Evaluators Recognize and Favor Their Own Generations (2024) · cites this paper as Mitchell et al., 2023
The Experiments section (page unavailable in our copy) describes several detection methods that use token-wise probabilities, including one based on average token-wise log probability; DetectGPT also uses log probabilities. This supports the broader point that probability information is important to these methods, but the paper does not identify perplexity itself as a crucial component of many methods. (LLM, conf=0.78)
Experimental — accuracy not yet measuredThe AUROC scores of DetectGPT drop from 96.5% before the attack to 59.8% after the attack.
“The AUROC scores of DetectGPT (Mitchell et al., 2023) drop from 96 . 5 % (before the attack) to 59 . 8 % (after the attack).”
From Can AI-Generated Text be Reliably Detected? (2023) · cites this paper as Mitchell et al., 2023
Figure 5 in Section 5.1 reports that DetectGPT’s AUROC generally declines as simulated human edits become heavier. It does not report a before-and-after attack comparison of 96.5% and 59.8%, so the specific figures and attack result are unsupported by our copy. (LLM, conf=0.79)
Experimental — accuracy not yet measuredMireshghallah et al. found that smaller language models like OPT-125M are better AIGC detectors than large language models like GPTJ-6B.
“Mireshghallah et al. [110] experimented to test whether other language models can be used to detect machine-generated text from one language model and found out that smaller language models like OPT-125M are better AIGC detectors than large language models like GPTJ-6B.”
From Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integrity (2024) · cites this paper as [110]
In §2 (Related Work; page unavailable), DetectGPT reports that Mireshghallah et al. found smaller models to be better proxy scoring models for perturbation-based detection when the generating model is unknown. It does not give the claimed OPT-125M versus GPTJ-6B comparison, so that specific example is not verified by our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measuredAfter altering the content using an Automated Paraphrasing tool (APT), the detection rate decreased to 4.6%.
“However, after altering the content using an Automated Paraphrasing tool (APT), the detection rate decreased to 4.6%.”
From Simple techniques to bypass GenAI text detectors: implications for inclusive education (2024) · cites this paper as Mitchell et al. (2023)
Section 5.2 reports that detection performance generally declines as machine-generated text is increasingly revised. It simulates revisions by replacing text spans with T5-3B output and reports AUROC, not a detection rate of 4.6% after use of an Automated Paraphrasing Tool. No page number is provided in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measured
DetectGPT correctly detected 70.3% of model-generated sequences from GPT2-XL.
“Mitchell et al. (2023) identified that DetectGPT correctly detected 70.3% of model-generated sequences from GPT2-XL.”
From Simple techniques to bypass GenAI text detectors: implications for inclusive education (2024) · cites this paper as Mitchell et al. (2023)
Table 1 reports that DetectGPT detects GPT-2-generated samples, but its results are AUROC values, not the percentage of generated sequences correctly detected. The cited paper’s results do not substantiate the specific 70.3% figure. (LLM, conf=0.82)
Experimental — accuracy not yet measuredNeural network-based detectors require retraining when encountering text generated by new LLMs to ensure reliable detection.
“These detectors require retraining when encountering text generated by new LLMs to ensure reliable detection [Mitchell et al., 2023;Chakraborty et al., 2023b].”
From Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges (2025) · cites this paper as Mitchell et al., 2023
Section 5.1 reports that supervised detectors can overfit to their training data, including the source model, and perform worse under distribution shift. Section 5 contrasts this with zero-shot detectors, which do not require retraining for new source models. The paper does not establish that neural network-based detectors invariably require retraining for every new LLM to ensure reliable detection. (LLM, conf=0.84)
Experimental — accuracy not yet measuredMachine-generated text detection methods remain limited in their precision and scope.
“However, these methods remain limited in their precision and scope.”
From On the Risk of Misinformation Pollution with Large Language Models (2023) · cites this paper as Mitchell et al., 2023
Mitchell et al. (2023) observe that AI-generated passages tend to lie in negative curvature of log probability of texts.
“Mitchell et al. (2023) observe that AI-generated passages tend to lie in negative curvature of log probability of texts.”
From Can AI-Generated Text be Reliably Detected? (2023) · cites this paper as Mitchell et al. (2023)
The majority of previous research on automatic detection of machine-generated text has focused on binary classification.
“The majority of previous research on automatic detection of machine-generated text has focused on binary classification [Jawahar et al., 2020b;Mitchell et al., 2023].”
From Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges (2025) · cites this paper as Mitchell et al., 2023
Feature-based methods from LLM-generated text detection, such as Log-likelihood, GLTR, and log-rank, are adapted to analyze and classify human-LLM co-authored texts versus those solely authored by humans or LLMs.
“To effectively analyze and classify human-LLM co-authored texts versus those solely authored by humans or LLMs, feature-based methods from the problem of LLM-generated text detection, such as Log-likelihood [Solaiman et al., 2019], GLTR [Gehrmann et al., 2019a], and log-rank [Mitchell et al., 2023], are adapted to this task.”
From Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges (2025) · cites this paper as Mitchell et al., 2023
Models like DNA-GPT and DetectGPT utilized the T5 model to simulate scenarios in which humans modify LLM-generated texts.
“Models like DNA-GPT [Yang et al., 2023a] and DetectGPT [Mitchell et al., 2023] utilized the T5 model [Raffel et al., 2020] to simulate scenarios in which humans modify LLMgenerated texts.”
From Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges (2025) · cites this paper as Mitchell et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 22
- Bakhtin, A., Gross, S., Ott, M., Deng, Y., Ranzato, M., and Szlam, A. Real or fake? Learning to discriminate machine from human generated text. arXiv, 2019. URL http://arxiv.org/abs/1906.03351. · arXiv
- Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., Pieler, M., Prashanth, U. S., Purohit, S., Reynolds, L., Tow, J., Wang, B., and Weinbach, S. GPT-NeoX-20B: An open-source autoregressive language model. In Pro- ceedings of the ACL Workshop on Challenges & Perspec- tives in Creating Large Language Models, 2022. URL https://arxiv.org/abs/2204.06745. · arXiv
- Bojar, O. r., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huck, M., Jimeno Yepes, A., Koehn, P., Logacheva, V., Monz, C., Negri, M., Neveol, A., Neves, M., Popel, M., Post, M., Rubino, R., Scarton, C., Specia, L., Turchi, M., Verspoor, K., and Zampieri, M. Findings of the 2016 conference on machine translation. In Pro- ceedings of the First Conference on Machine Translation, pp. 131-198, Berlin, Germany, August 2016. Associa- tion for Computational Linguistics. URL http://www. aclweb.org/anthology/W/W16/W16-2301.
- Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1877-1901.
- Curran Asso- ciates, Inc., 2020. URL https://proceedings. neurips.cc/paper/2020/file/ 1457c0d6bfcb4967418bfb8ac142f64a-Paper. pdf.
- Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Shazeer, N., Prabhakaran, V., Reif, E., Du, N., Hutchinson, B., Pope, R., Bradbury, J., Austin, J., Isard, M., Gur-Ari, G., Yin, P., Duke, T., Levskaya, A., Ghemawat, S., Dev, S., Michalewski, H., Garcia, X., Misra, V., Robinson, K., Fedus, L., Zhou, D., Ippolito, D., Luan, D., Lim, H., Zoph, B., Spiridonov, A., Sepassi, R., Dohan, D., Agrawal, S., Omernick, M., Dai, A. M., Pillai, T. S., Pellat, M., Lewkowycz, A., Moreira, E., Child, R., Polo- zov, O., Lee, K., Zhou, Z., Wang, X., Saeta, B., Diaz, M., Firat, O., Catasta, M., Wei, J., Meier-Hellstern, K., Eck, D., Dean, J., Petrov, S., and Fiedel, N. PaLM: Scaling language modeling with pathways, 2022. URL https://arxiv.org/abs/2204.02311. · arXiv
- Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. Deep reinforcement learning from human preferences. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30.
- Curran Associates, Inc., 2017. URL https://proceedings.neurips. cc/paper_files/paper/2017/file/ d5e2c0adad503c91f91df240d0cd4e49-Paper. pdf.
- Guarnera, L., Giudice, O., and Battiato, S. Deepfake detec- tion by analyzing convolutional traces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020.
- Jawahar, G., Abdul-Mageed, M., and Lakshmanan, L. V. S. Automatic detection of machine generated text: A critical survey. In International Conference on Computational Linguistics, 2020.
- Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., and Goldstein, T. A watermark for large language mod- els, 2023. URL https://arxiv.org/abs/2301. 10226.
- Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., and Zou, J. Gpt detectors are biased against non-native english writers. arXiv preprint arXiv:2304.02819, 2023. · arXiv
- Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019. · arXiv
- Mireshghallah, F., Mattern, J., Gao, S., Shokri, R., and Berg- Kirkpatrick, T. Smaller language models are better black- box machine-generated text detectors. arXiv preprint arXiv:2305.09859, 2023. · arXiv
- Narayan, S., Cohen, S. B., and Lapata, M. Don't give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization. In Proceed- ings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 2018.
- Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Ex- ploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020. URL http://jmlr. org/papers/v21/20-074.html.
- Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., and Feizi, S. Can ai-generated text be reliably de- tected? arXiv preprint arXiv:2303.11156, 2023. · arXiv
- Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert- Voss, A., Wu, J., Radford, A., and Wang, J. Release strategies and the social impacts of language models, 2019. URL https://arxiv.org/ftp/arxiv/ papers/1908/1908.09203.pdf.
- Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., and Choi, Y. Defending against neural fake news. In Neural Information Processing Systems, 2019.
- Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., Mi- haylov, T., Ott, M., Shleifer, S., Shuster, K., Simig, D., Koura, P. S., Sridhar, A., Wang, T., and Zettlemoyer, L. Opt: Open pre-trained transformer language mod- els, 2022. URL https://arxiv.org/abs/2205. 01068.
- Zhao, H., Zhou, W., Chen, D., Wei, T., Zhang, W., and Yu, N. Multi-attentional deepfake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2185-2194, 2021.
- Zi, B., Chang, M., Chen, J., Ma, X., and Jiang, Y.-G. Wild- deepfake: A challenging real-world dataset for deepfake detection. In Proceedings of the 28th ACM international conference on multimedia, pp. 2382-2390, 2020.
Cited by
Sorted by citations
Regulating ChatGPT and other Large Generative AI Models
Hacker, P., Engel, A., & Mauer, M. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594067 · Cited by 466 · see citing works
“This should be coupled with research on AI-content detection to highlight such output where watermarks fail [121, 140].”
BackgroundA Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335 · see citing works
Large AI Models in Health Informatics: Applications, Challenges, and the Future
Qiu, J., Li, L., Sun, J., Peng, J., Shi, P., Zhang, R., Dong, Y., Lam, K., Lo, F. P.-W., Xiao, B. J., et al. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3316750 · Cited by 246 · see citing works
An Empirical Study of the Non-Determinism of ChatGPT in Code Generation
Ouyang, S., Zhang, J. M., Harman, M., & Wang, M. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3697010 · Cited by 258 · see citing works
“The randomness in LLMs typically comes from the sampling methods used during text generation, such as top-k sampling or nucleus sampling [31, 49].”
Sohail, S. S., Farhat, F., Himeur, Y., Nadeem, M., Madsen, D. Ø., Singh, Y., Atalla, S., & Mansoor, W. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.101675 · Cited by 192 · see citing works
“However, the former step, which focuses on specifically identifying texts generated by ChatGPT, is relatively new, and researchers are actively working to address this issue (Dugan et al., 2022; Mitchell et al., 2023; Curtis, 2023; Mitrovic et al., 2023).”
Terwiesch, C., Meincke, L., Girotra, K., Mollick, E., Nave, G., & Ulrich, K. T. (2026) · Production and Operations Management · doi:10.1177/10591478261474243 · Cited by 191 · see citing works
Terwiesch, C., Meincke, L., Girotra, K., Mollick, E. R., Nave, G., & Ulrich, K. T. (2023) · doi:10.2139/ssrn.4526071 · Cited by 174 · see citing works
Sison, A. J. G., Daza, M. T., Gozalo-Brizuela, R., & Garrido-Merchán, E. C. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2225931 · Cited by 169 · see citing works
Can AI-Generated Text be Reliably Detected?
Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11156 · Cited by 161 · see citing works
“In this section, we discuss paraphrasing attacks that can degrade type-II errors of state-of-the-art AI text detectors such as soft watermarking [1], zero-shot detectors [2], trained neural network-based detectors [3], and retrieval-based detectors [4].”
BackgroundMethodInfluentialDefending ChatGPT against jailbreak attack via self-reminders
Xie, Y., Yi, J., Shao, J., Curl, J., Lyu, L., Chen, Q., Xie, X., & Wu, F. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00765-8 · Cited by 158 · see citing works
Ali, K., Barhom, N., Tamimi, F., & Duggal, M. (2024) · European Journal of Dental Education · doi:10.1111/eje.12937 · Cited by 152 · see citing works
A Survey on the Detection and Impacts of Deepfakes in Visual, Audio, and Textual Formats
Mubarak, R., Alsboui, T., Alshaikh, O., Inuwa-Dutse, I., Khan, S., & Parkinson, S. (2023) · IEEE Access · doi:10.1109/access.2023.3344653 · Cited by 132 · see citing works
Desaire, H., Chua, A. E., Isom, M., Jarošová, R., & Hua, D. (2023) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2023.101426 · Cited by 122 · see citing works
Simple techniques to bypass GenAI text detectors: implications for inclusive education
Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., McGaughran, J., & Khuat, H. Q. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00487-w · Cited by 117 · see citing works
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text
Su, J., Zhuo, T. Y., Wang, D., & Nakov, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.827 · Cited by 105 · see citing works
MGTBench: Benchmarking Machine-Generated Text Detection
He, X., Shen, X., Chen, Z., Michael Backes, A., & Zhang, Y. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3670344 · Cited by 105 · see citing works
Haemmerli, J., Sveikata, L., Nouri, A., May, A. T., Egervari, K. L., Freyschlag, C. F., Lobrinus, J. A., Migliorini, D., Momjian, S., Sanda, N., et al. (2023) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2023-100775 · Cited by 99 · see citing works
An overview of fake news detection: From a new perspective
Hu, B., Mao, Z., & Zhang, Y. (2024) · Fundamental Research · doi:10.1016/j.fmre.2024.01.017 · Cited by 90 · see citing works
Self-consuming generative models go MAD
Casco-Rodriguez, J., Alemohammad, S., Luzi, L., Imtiaz, A., Babaei, H., LeJeune, D., Siahkoohi, A., & Baraniuk, R. (2023) · LatinX in AI at Neural Information Processing Systems Conference 2023 · doi:10.52591/lxai202312101 · Cited by 83 · see citing works
Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods
Fraser, K., Dawkins, H., & Kiritchenko, S. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.16665 · Cited by 80 · see citing works
MAGE: Machine-generated Text Detection in the Wild
Li, Y., Li, Q., Cui, L., Bi, W., Wang, Z., Wang, L., Yang, L., Shi, S., & Zhang, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.3 · Cited by 79 · see citing works
The impact of generative AI on information processing: Evidence from the ban of ChatGPT in Italy
Bertomeu, J., Lin, Y., Liu, Y., & Ni, Z. (2025) · Journal of Accounting and Economics · doi:10.1016/j.jacceco.2025.101782 · Cited by 74 · see citing works
Anatomy of an AI-powered malicious social botnet
Yang, K., & Menczer, F. (2024) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2024.icwsm.7 · Cited by 71 · see citing works
Pudasaini, S., Miralles-Pechuán, L., Lillis, D., & Llorens Salvador, M. (2025) · Journal of Academic Ethics · doi:10.1007/s10805-024-09576-x · Cited by 70 · see citing works
Liang, W., Izzo, Z., Zhang, Y., Lepp, H., Cao, H., Zhao, X., Chen, L., Ye, H., Liu, S., Huang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07183 · Cited by 73 · see citing works
“More recently, Detect-GPT (Mitchell et al., 2023a) suggests that AI-generated text typically occupies regions with negative log probability curvature.”
On the Risk of Misinformation Pollution with Large Language Models
Pan, Y., Pan, L., Chen, W., Nakov, P., Kan, M.-Y., & Wang, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.97 · Cited by 70 · see citing works
Evading Watermark based Detection of AI-Generated Content
Jiang, Z., Zhang, J., & Gong, N. Z. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3623189 · Cited by 67 · see citing works
Ibrahim, K. (2023) · Language Testing in Asia · doi:10.1186/s40468-023-00260-2 · Cited by 66 · see citing works
Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text?
Bhattacharjee, A., & Liu, H. (2024) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3655103.3655106 · Cited by 65 · see citing works
Detecting ChatGPT-Generated Code Submissions in a CS1 Course Using Machine Learning Models
Hoq, M., Shi, Y., Leinonen, J., Babalola, D., Lynch, C. F., Price, T., & Akram, B. (2024) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3626252.3630826 · Cited by 65 · see citing works
Identifying and Mitigating the Security Risks of Generative AI
Barrett, C., Boyd, B., Bursztein, E., Carlini, N., Chen, B., Choi, J., Chowdhury, A. R., Christodorescu, M., Datta, A., Feizi, S., et al. (2023) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000041 · Cited by 57 · see citing works
Self-citationAn Empirical Study of AI-Generated Text Detection Tools
Akram, A. (2023) · Advances in Machine Learning & Artificial Intelligence · doi:10.33140/amlai.04.02.03 · Cited by 57 · see citing works
Parray, A. A., Inam, Z. M., Ramonfaur, D., Haider, S. S., Mistry, S. K., & Pandya, A. K. (2023) · Global Transitions · doi:10.1016/j.glt.2023.05.001 · Cited by 60 · see citing works
Koike, R., Kaneko, M., & Okazaki, N. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30120 · Cited by 55 · see citing works
Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models
Xu, Z., & Sheng, V. S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i21.30361 · Cited by 54 · see citing works
SeqXGPT: Sentence-Level AI-Generated Text Detection
Wang, P., Li, L., Ren, K., Jiang, B., Zhang, D., & Qiu, X. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.73 · Cited by 53 · see citing works
Synchronizing Object Detection: Applications, Advancements and Existing Challenges
Hosain, M. T., Zaman, A., Abir, M. R., Akter, S., Mursalin, S., & Khan, S. S. (2024) · IEEE Access · doi:10.1109/access.2024.3388889 · Cited by 52 · see citing works
The Looming Threat of Fake and LLM-generated LinkedIn Profiles
Ayoobi, N., Shahriar, S., & Mukherjee, A. (2023) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3603163.3609064 · Cited by 48 · see citing works
Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts
Tulchinskii, E., Kuznetsov, K., Kushnareva, L., Cherniavskii, D., Nikolenko, S., Burnaev, E. V., Barannikov, S., & Piontkovskaya, I. (2023) · neural information processing systems · doi:10.52202/075280-1706 · Cited by 49 · see citing works
Shah, A., Ranka, P., Dedhia, U., Prasad, S., Muni, S., & Bhowmick, K. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.01410110 · Cited by 47 · see citing works
Fighting disinformation with artificial intelligence: fundamentals, advances and challenges
Montoro-Montarroso, A., Cantón-Correa, J., Rosso, P., Chulvi, B., Panizo-LLedot, Á., Huertas‐Tato, J., Figueras, B. C., Rementeria, M. J., & Gómez‐Romero, J. (2023) · El Profesional de la Informacion · doi:10.3145/epi.2023.may.22 · Cited by 45 · see citing works
Liu, X., Zhang, Z., Wang, Y., Pu, H., Lan, Y., & Shen, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1005 · Cited by 44 · see citing works
LLMDet: A Third Party Large Language Models Generated Text Detection Tool
Wu, K., Pang, L., Shen, H., Cheng, X., & Chua, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.139 · Cited by 45 · see citing works
ConDA: Contrastive Domain Adaptation for AI-generated Text Detection
Bhattacharjee, A., Kumarage, T., Moraffah, R., & Liu, H. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.40 · Cited by 43 · see citing works
Comprehensive Review of Natural Language Processing (NLP) in Vascular Surgery
Lareyre, F., Nasr, B., Chaudhuri, A., Lorenzo, G. D., Carlier, M., & Raffort, J. (2023) · EJVES Vascular Forum · doi:10.1016/j.ejvsvf.2023.09.002 · Cited by 42 · see citing works
Dianova, V. G., & Schultz, M. D. (2023) · Industry and Higher Education · doi:10.1177/09504222231199989 · Cited by 43 · see citing works
A Survey on Detection of LLMs-Generated Content
Yang, X., Pan, L., Zhao, X., Chen, H., Petzold, L. R., Wang, W. Y., & Cheng, W. D. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.572 · Cited by 41 · see citing works
GPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · arXiv · doi:10.48550/arxiv.2304.02819 · Cited by 40 · see citing works
Generative AI models should include detection mechanisms as a condition for public release
Knott, A., Pedreschi, D., Chatila, R., Chakraborti, T., Leavy, S., Baeza-Yates, R. A., Eyers, D. M., Trotman, A., Teal, P. D., Biecek, P., et al. (2023) · Ethics and Information Technology · doi:10.1007/s10676-023-09728-4 · Cited by 40 · see citing works
Three Bricks to Consolidate Watermarks for Large Language Models
Fernandez, P., Chaffin, A., Tit, K., Chappelier, V., & Furon, T. (2023) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs58808.2023.10374576 · Cited by 39 · see citing works
Red Teaming Language Model Detectors with Language Models
Shi, Z., Wang, Y., Yin, F., Chen, X., Chang, K.-W., & Hsieh, C. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00639 · Cited by 39 · see citing works
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
Kumar, S., Balachandran, V., Njoo, L., Anastasopoulos, A., & Tsvetkov, Y. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.241 · Cited by 38 · see citing works
Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges
Huang, B., Chen, C., & Shu, K. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3715073.3715076 · Cited by 42 · see citing works
“These detectors require retraining when encountering text from new LLMs to ensure reliable detection (Chakraborty et al., 2023b, Mitchell et al., 2023).”
InfluentialMultitude: Large-scale multilingual machine-generated text detection benchmark
Macko, D., Moro, R., Uchendu, A., Lucas, J., Yamashita, M., Pikuliak, M., Srba, I., Le, T., Lee, D., Simko, J., et al. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.616 · Cited by 38 · see citing works
Najafali, D., Reiche, E., Camacho, J. M., Morrison, S. D., & Dorafshar, A. H. (2023) · Aesthetic Surgery Journal · doi:10.1093/asj/sjad056 · Cited by 38 · see citing works
Hanley, H. W. A., & Durumeric, Z. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31333 · Cited by 36 · see citing works
Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts?
Uchendu, A., Lee, J., Shen, H., Le, T., Huang, T.-H., & Lee, D. (2023) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v11i1.27557 · Cited by 35 · see citing works
Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective
Uchendu, A., Le, T., & Lee, D. (2023) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3606274.3606276 · Cited by 34 · see citing works
Large Language Models for Conducting Advanced Text Analytics Information Systems Research
Ampel, B., Yang, C.-H., Hu, J. L., & Chen, H. (2024) · ACM Transactions on Management Information Systems · doi:10.1145/3682069 · Cited by 31 · see citing works
Token Prediction as Implicit Classification to Identify LLM-Generated Text
Chen, Y., Kang, H., Zhai, V., Li, L., Singh, R., & Raj, B. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.810 · Cited by 30 · see citing works
A Robust Semantics-based Watermark for Large Language Model against Paraphrasing
Ren, J., Xu, H., Liu, Y., Cui, Y., Wang, S., Yin, D., & Tang, J. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.40 · Cited by 30 · see citing works
GPT-who: An Information Density-based Machine-Generated Text Detector
Venkatraman, S., Uchendu, A., & Lee, D. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.8 · Cited by 30 · see citing works
Publicly-Detectable Watermarking for Language Models
Fairoze, J., Garg, S., Jha, S., Mahloujifar, S., Mahmoody, M., & Wang, M. (2025) · IACR Communications in Cryptology · doi:10.62056/ahmpdkp10 · Cited by 29 · see citing works
Malik, M. A., & Amjad, A. I. (2025) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2025.8.1.9 · Cited by 28 · see citing works
Fu, Y., Xiong, D., & Dong, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29756 · Cited by 27 · see citing works
Evaluating AI and human authorship quality in academic writing through physics essays
Yeadon, W., Agra, E., Inyang, O.-O., Mackay, P., & Mizouri, A. (2024) · European Journal of Physics · doi:10.1088/1361-6404/ad669d · Cited by 26 · see citing works
Enhancing Machine-Generated Text Detection: Adversarial Fine-Tuning of Pre-Trained Language Models
Lee, D., & Jang, B. (2024) · IEEE Access · doi:10.1109/access.2024.3396820 · Cited by 26 · see citing works
Classification of human- and AI-generated texts for different languages and domains
Schaaff, K., Schlippe, T., & Mindner, L. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-024-10143-3 · Cited by 26 · see citing works
Distinguishing LLM-Generated from Human-Written Code by Contrastive Learning
Xu, X., Ni, C., Guo, X., Liu, S., Wang, X., Liu, K., & Yang, X. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3705300 · Cited by 26 · see citing works
AI-generated content in cross-domain applications: Research trends, challenges and propositions
Li, J., Qu, L., Cai, T., Zhao, Z., Haldar, N. A. H., Krishna, A., Kong, X., Macau, F., Chakraborty, T., Deroy, A., et al. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114634 · Cited by 26 · see citing works
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
Zeng, Z., Sha, L., Li, Y., Yang, K., Gašević, D., & Chen, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30258 · Cited by 25 · see citing works
M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection
Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Mohammed Afzal, O., Mahmoud, T., Puccetti, G., Arnold, T., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.218 · Cited by 25 · see citing works
Beyond Black Box AI generated Plagiarism Detection: From Sentence to Document Level
Quidwai, A., Li, C., & Dube, P. (2023) · Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) · doi:10.18653/v1/2023.bea-1.58 · Cited by 24 · see citing works
ChatGPT Code Detection: Techniques for Uncovering the Source of Code
Oedingen, M., Engelhardt, R. C., Denz, R., Hammer, M., & Konen, W. (2024) · AI · doi:10.3390/ai5030053 · Cited by 24 · see citing works
Detection of AI-Generated Text Using Large Language Model
Prajapati, M., Baliarsingh, S. K., Dora, C., Bhoi, A., Hota, J., & Mohanty, J. P. (2024) · International Conference on Emerging Systems and Intelligent Computing (ESIC) · doi:10.1109/esic60604.2024.10481602 · Cited by 24 · see citing works
Detection of Fake Generated Scientific Abstracts
Theocharopoulos, P. C., Anagnostou, P., Tsoukala, A., Georgakopoulos, S. V., Tasoulis, S. K., & Plagianakos, V. P. (2023) · IEEE International Conference on Big Data Computing Service and Applications (BigDataService) · doi:10.1109/bigdataservice58306.2023.00011 · Cited by 23 · see citing works
Pratama, A. R. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2953 · Cited by 22 · see citing works
WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models
Tu, S., Sun, Y., Bai, Y., Yu, J., Hou, L., & Li, J. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.83 · Cited by 22 · see citing works
LegalLens: Leveraging LLMs for Legal Violation Identification in Unstructured Text
Bernsohn, D., Semo, G., Vazana, Y., Hayat, G., Hagag, B., Niklaus, J., Saha, R., & Truskovskyi, K. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.130 · Cited by 21 · see citing works
AI-generated text detection: A comprehensive review of methods, datasets, and applications
Kehkashan, T., Riaz, R. A., Al‐Shamayleh, A. S., Akhunzada, A., Ali, N., Hamza, M., & Akbar, F. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100793 · Cited by 21 · see citing works
AI Usage Cards: Responsibly Reporting AI-Generated Content
Wahle, J. P., Ruas, T., Mohammad, S. M., Meuschke, N., & Gipp, B. (2023) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl57899.2023.00060 · Cited by 20 · see citing works
AI-Powered Steganography: Advances in Image, Linguistic, and 3D Mesh Data Hiding – A Survey
Setiadi, D. R. I. M., Ghosal, S. K., & Sahu, A. K. (2025) · Journal of Future Artificial Intelligence and Technologies · doi:10.62411/faith.3048-3719-76 · Cited by 20 · see citing works
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection
Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Whitehouse, C., Afzal, O. M., Mahmoud, T., Sasaki, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.14902 · Cited by 19 · see citing works
Detection Vs. Anti-detection: Is Text Generated by AI Detectable?
Zhang, Y., Ma, Y., Liu, J., Liu, X., Wang, X., & Lu, W. P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-57850-2_16 · Cited by 19 · see citing works
Li, X., Ruan, F., Wang, H., Long, Q., & Su, W. (2025) · The Annals of Statistics · doi:10.1214/24-aos2468 · Cited by 19 · see citing works
LLM Evaluators Recognize and Favor Their Own Generations
Panickssery, A., Bowman, S. R., & Feng, S. (2024) · arXiv · doi:10.48550/arxiv.2404.13076 · Cited by 18 · see citing works
“In particular, the detector LLM is not given explicit access to information such as perplexity, which in many detection methods is a crucial component (Mitchell et al., 2023; Hans et al., 2024).”
LLM-as-a-Coauthor: Can Mixed Human-Written and Machine-Generated Text Be Detected?
Zhang, Q., Gao, C., Chen, D., Huang, Y., Huang, Y., Sun, Z., Zhang, S., Li, W., Fu, Z., Wan, Y., et al. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.29 · Cited by 18 · see citing works
How to Detect AI-Generated Texts?
Nguyen, T. T., Hatua, A., & Sung, A. H. (2023) · IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) · doi:10.1109/uemcon59035.2023.10316132 · Cited by 18 · see citing works
On the Zero-Shot Generalization of Machine-Generated Text Detectors
Xiao, P., Zhang, J., Han, X., Tsvetkov, Y., & He, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.318 · Cited by 18 · see citing works
Pitfalls (and advantages) of sophisticated large language models
Strasser, A. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-18851-0.00007-x · Cited by 18 · see citing works
Watermarking LLMs with Weight Quantization
Li, L., Jiang, B., Wang, P., Ren, K., Yan, H., & Qiu, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.220 · Cited by 17 · see citing works
PostMark: A Robust Blackbox Watermark for Large Language Models
Chang, Y., Krishna, K., Houmansadr, A., Wieting, J., & Iyyer, M. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.506 · Cited by 17 · see citing works
On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook
Fan, M., Wang, C., Chen, C., Liu, Y., & Huang, J. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02375-w · Cited by 17 · see citing works
Use prompt to differentiate text generated by ChatGPT and humans
An, R., Yang, Y., Yang, F., & Wang, S. (2023) · Machine Learning with Applications · doi:10.1016/j.mlwa.2023.100497 · Cited by 17 · see citing works
Beat LLMs at their own game: Zero-shot LLM-generated text detection via querying ChatGPT
Zhu, B., Yuan, L., Cui, G., Chen, Y., Fu, C., He, B., Deng, Y., Liu, Z., Sun, M., & Gu, M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.463 · Cited by 16 · see citing works
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future
Yang, L., Song, Y., Ren, X., Lyu, C., Wang, Y., Zhuo, J., Liu, L., Wang, J., Foster, J., & Zhang, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.276 · Cited by 16 · see citing works
LLM-based Attack Scenarios Generator with IT Asset Management and Vulnerability Information
Naito, T., Watanabe, R., & Mitsunaga, T. (2023) · International Conference on Signal Processing and Information Security (ICSPIS) · doi:10.1109/icspis60075.2023.10344019 · Cited by 16 · see citing works
Quian, A., & García, J. S. (2024) · Revista de Comunicación · doi:10.26441/rc23.1-2024-3374 · Cited by 16 · see citing works
Sufi, F. K. (2024) · Journal of Economy and Technology · doi:10.1016/j.ject.2024.09.001 · Cited by 16 · see citing works
Evaluating the performance and robustness of LLMs in materials science Q&A and property predictions
Wang, H., Li, K., Ramsay, S., Fehlis, Y., Kim, E., & Hattrick-Simpers, J. R. (2025) · Digital Discovery · doi:10.1039/d5dd00090d · Cited by 16 · see citing works
- Authors
- Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., & Finn, C.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2301.11305
- OpenAlex
- W4318351452
- Cited by
- 151
- References
- 0
- Citations by year
- 2026: 7 · 2025: 32 · 2024: 51 · 2023: 60 · 2021: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- AI-generated text detection, large language models, zero-shot detection, fake news detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
