The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis
Open accessAbstract
As the use of Large Language Models (LLMs) in text generation tasks proliferates, concerns arise over their potential to compromise academic integrity. The education sector currently tussles with distinguishing student-authored homework assignments from AI-generated ones. This paper addresses the challenge by introducing HowkGPT, designed to identify homework assignments generated by AI. HowkGPT is built upon a dataset of academic assignments and accompanying metadata [17] and employs a pretrained LLM to compute perplexity scores for student-authored and ChatGPT-generated responses. These scores then assist in establishing a threshold for discerning the origin of a submitted assignment. Given the specificity and contextual nature of academic work, HowkGPT further refines its analysis by defining category-specific thresholds derived from the metadata, enhancing the precision of the detection. This study emphasizes the critical need for effective strategies to uphold academic integrity amidst the growing influence of LLMs and provides an approach to ensuring fair and accurate grading in educational institutions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
HowkGPT detects LLM-generated text, specifically homework assignments, by calculating and comparing perplexity scores derived from student-written and ChatGPT-generated text.
“HowkGPT (Vasilatos et al. 2023), discerns LLM-generated text, specifically homework assignments, by calculating and comparing perplexity scores derived from studentwritten and ChatGPT-generated text.”
From A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions (2025) · cites this paper as Vasilatos et al. 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GLTR: Statistical Detection and Visualization of Generated Text
Gehrmann, S., Strobelt, H., & Rush, A. M. (2019) · arXiv · doi:10.48550/arxiv.1906.04043 · Cited by 460
Flagged first, then by citations
Research works printed but not linked: 18
- Christopher D. Brown and Herbert T. Davis. "Receiver operating characteristics curves and related decision measures: A tutorial". In: Chemometrics and Intelligent Laboratory Systems 80.1 (2006), pp. 24-38. ISSN: 0169- 7439. DOI: https://doi.org/10.1016/j.chemolab.2005. 05.004. URL: https://www.sciencedirect.com/science/ article/pii/S0169743905000766. · DOI
- Asli Celikyilmaz, Elizabeth Clark, and Jianfeng Gao. Evaluation of Text Generation: A Survey. 2021. arXiv: 2006.14799 [cs.CL]. · arXiv
- Suchin Gururangan et al. Don't Stop Pretraining: Adapt Language Models to Domains and Tasks. 2020. arXiv: 2004.10964 [cs.CL]. · arXiv
- Tatsunori B. Hashimoto, Hugh Zhang, and Percy Liang. Unifying Human and Statistical Evaluation for Natu- ral Language Generation. 2019. arXiv: 1904 . 02792 [cs.CL].
- Hazem Ibrahim et al. Perception, performance, and detectability of conversational artificial intelligence across 32 university courses. 2023. arXiv: 2305.13934 [cs.CY]. · arXiv
- Daphne Ippolito et al. Automatic Detection of Gener- ated Text is Easiest when Humans are Fooled. 2020. arXiv: 1911.00650 [cs.CL]. · arXiv
- Neel Kant et al. Practical Text Classification With Large Pre-Trained Language Models. 2018. arXiv: 1812 . 01207 [cs.CL].
- Mohammad Khalil and Erkan Er. Will ChatGPT get you caught? Rethinking of Plagiarism Detection. 2023. arXiv: 2302.04335 [cs.AI]. · arXiv
- David R Krathwohl. "A revision of Bloom's taxonomy: An overview". In: Theory into practice 41.4 (2002), pp. 212-218.
- Linyang Li et al. Origin Tracing and Detecting of LLMs. 2023. arXiv: 2304.14072 [cs.CL]. · arXiv
- Peng Liu, Lemei Zhang, and Jon Atle Gulla. Pre- train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systems. 2023. arXiv: 2302 . 03735 [cs.IR].
- Eric Mitchell et al. DetectGPT: Zero-Shot Machine- Generated Text Detection using Probability Curvature. 2023. arXiv: 2301.11305 [cs.CL]. · arXiv
- Sandra Mitrović, Davide Andreoletti, and Omran Ay- oub. ChatGPT or Human? Detect and Explain. Ex- plaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text. 2023. arXiv: 2301.13852 [cs.CL]. · arXiv
- Travis Munyer and Xin Zhong. DeepTextMark: Deep Learning based Text Watermarking for Detection of Large Language Model Generated Text. 2023. arXiv: 2305.05773 [cs.MM]. · arXiv
- Vinu Sankar Sadasivan et al. Can AI-Generated Text be Reliably Detected? 2023. arXiv: 2303.11156 [cs.CL]. · arXiv
- Ruixiang Tang, Yu-Neng Chuang, and Xia Hu. The Sci- ence of Detecting LLM-Generated Texts. 2023. arXiv: 2303.07205 [cs.CL]. · arXiv
- David Vilar et al. Prompting PaLM for Translation: Assessing Strategies and Performance. 2022. arXiv: 2211.09102 [cs.CL]. · arXiv
- Ashima Yadav and Dinesh Kumar Vishwakarma. "Sen- timent analysis using deep learning architectures: a review". In: Artificial Intelligence Review 53.6 (2020), pp. 4335-4385.
Cited by
Sorted by citations
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
Hans, A., Schwarzschild, A., Cherepanova, V., Kazemi, H., Saha, A., Goldblum, M., Geiping, J., & Goldstein, T. (2024) · arXiv · doi:10.48550/arxiv.2401.12070 · Cited by 24
Metadata
- Authors
- Vasilatos, C., Alam, M., Rahwan, T., Zaki, Y., & Maniatakos, M.
- Affiliations
- New York University Abu Dhabi (AE)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2305.18226
- OpenAlex
- W4378771612
- Cited by
- 34
- References
- 1
- Citations by year
- 2026: 8 · 2025: 12 · 2024: 13 · 2023: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- AI-generated text detection, ChatGPT, academic integrity, student homework, perplexity, large language models, context-aware detection, automated grading, university students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
