The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Assessing AI Detectors in Identifying AI-Generated Code: Implications for Education
Open accessAbstract
Educators are increasingly concerned about the usage of Large Language Models (LLMs) such as ChatGPT in programming education, particularly regarding the potential exploitation of imperfections in Artificial Intelligence Generated Content (AIGC) Detectors for academic misconduct.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Increased uses of AI in higher education are leading to a reevaluation of core assessment approaches [4,8,25].”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [25]
In the citing paper
“This exploitation concerns scholarly integrity and the legitimacy of assignments submitted by students, especially where detection tools are not yet proven highly reliable or limited to a specific domain [8,25].”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [25]
In the citing paper
“Traditional forms, including untimed take-home essays or standardized problem sets, grow increasingly vulnerable to exploitation by AI products of the sort of ChatGPT.”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [25]
In the citing paper
“A central issue is the risk of false accusations when these tools incorrectly flag human-written work as AI-generated [8,25].”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [25]
In the citing paper
“Rapidly integrating large language models (LLMs) like ChatGPT into academic environments requires reliable detection tools to distinguish between AI-generated and human-written content [25].”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [25]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
BAİDOO-ANU, D., & OWUSU ANSAH, L. (2023) · Journal of AI · doi:10.61969/jai.1337500 · Cited by 3,939
Baidoo-Anu, D., & Owusu Ansah, L. (2023) · doi:10.2139/ssrn.4337484 · Cited by 3,004
A comprehensive AI policy education framework for university teaching and learning
Chan, C. K. Y. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00408-3 · Cited by 1,577
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2023) · arXiv · doi:10.48550/arxiv.2307.06435 · Cited by 1,259
Grassini, S. (2023) · Education Sciences · doi:10.3390/educsci13070692 · Cited by 1,048
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029
Self-citationClassification Model Evaluation Metrics
Vujovic, Ž. Ð. (2021) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2021.0120670 · Cited by 620 · cited 1×
“For this empirical study, we have selected a few metrics [23, 34] to evaluate the performance of selected AIGC Detectors.”
MethodChatGPT and the rise of generative AI: Threat to academic integrity?
Eke, D. O. (2023) · Journal of Responsible Technology · doi:10.1016/j.jrt.2023.100060 · Cited by 532
GLTR: Statistical Detection and Visualization of Generated Text
Gehrmann, S., Strobelt, H., & Rush, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/p19-3019 · Cited by 470 · cited 1×
“GPTZero [1], Sapling [2], GPT-2 Detector [32], DetectGPT [7, 25], and Giant Language Model Test Room (GLTR) [17, 19, 28] offer educators reliable tools to preserve academic integrity and encourage ethical AI usage.”
BackgroundGLTR: Statistical Detection and Visualization of Generated Text
Gehrmann, S., Strobelt, H., & Rush, A. M. (2019) · arXiv · doi:10.48550/arxiv.1906.04043 · Cited by 460 · cited 1×
“GPTZero [1], Sapling [2], GPT-2 Detector [32], DetectGPT [7, 25], and Giant Language Model Test Room (GLTR) [17, 19, 28] offer educators reliable tools to preserve academic integrity and encourage ethical AI usage.”
Background
Research works printed but not linked: 15
- Fawad Ali. 2023. GPT-1 to GPT-4: Each of OpenAI's GPT Models Explained and Compared. (11 April 2023). https://www.makeuseof.com/gpt-models-explained-and-compared/
- Cecilia Ka Yuk Chan and Katherine KW Lee. 2023. The AI generation gap: Are Gen Z students more interested in adopting generative AI such as ChatGPT in teaching and learning than their Gen X and Millennial Generation teachers? arXiv preprint arXiv:2305.02878 (2023). · arXiv
- Hailin Chen, Fangkai Jiao, Xingxuan Li, Chengwei Qin, Mathieu Ravaut, Ruochen Zhao, Caiming Xiong, and Shafiq Joty. 2023. ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up? arXiv preprint arXiv:2311.16989 (2023). · arXiv
- Frances Chumney. 2018. PAIRED SAMPLES t & WILCOXON SIGNED RANKS TESTS. Retrieved January 24 (2018), 2022.
- Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection. arXiv preprint arxiv:2301.07597 (2023). · arXiv
- Ajay Kulkarni, Deri Chong, and Feras A Batarseh. 2020. Foundations of data imbalance and solutions for a data democracy. In Data democracy. Elsevier, 83--106.
- Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. In Proceedings of the 40th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 202), Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.). PMLR, 24950--24962. https://proceedings.mlr.press/v202/mitchell23a.html
- Jahna Otterbacher. 2023. Why technical solutions for detecting AI-generated content in research and education are insufficient. Patterns 4, 7 (2023).
- Jiahong Su and Weipeng Yang. 2023. Unlocking the power of ChatGPT: A framework for applying generative AI in education. ECNU Review of Education (2023), 20965311231168423.
- Teo Susnjak. 2022. ChatGPT: The end of online exam integrity? arXiv preprint arXiv:2212.09292 (2022). · arXiv
- Levent Uzun. 2023. ChatGPT and academic integrity concerns: Detecting artificial intelligence generated content. Language Education and Technology 3, 1 (2023).
- Jian Wang, Shangqing Liu, Xiaofei Xie, and Yi Li. 2023. Evaluating AIGC Detectors on Code Content. arXiv preprint arXiv:2304.05193 (2023). · arXiv
- www.quescol.com. 2023. Python Coding Question: 90+ Python Interview Coding Questions. https://quescol.com/interview-preparations/python-coding-question#google_vignette Last accessed on May 16, 2023.
- www.quescol.com. 2023. Quescol - A Platform That Provides Previous Year Questions And Answers. https://quescol.com/ Last accessed on Dec 23, 2023.
- Franco Zambonelli and H Van Dyke Parunak. 2002. Signs of a revolution in computer science and software engineering. In International Workshop on Engineering Societies in the Agents World. Springer, 13--28.
Cited by
Heilala, V., Araya, R., & Hämäläinen, R. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707764 · Cited by 68
Gaitantzi, A., & Kazanidis, I. (2025) · Applied Sciences · doi:10.3390/app15073960 · Cited by 53
Humble, N. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00494-x · Cited by 55
“Included papers highlighted that misconduct with GAI can be very difficult to detect (Jalil et al., 2023; Pan et al., 2024; Quille et al., 2024; Sağlam et al., 2024; Singh et al., 2023; Xu et al., 2024; Zeng et al., 2024) and that educators therefore should avoid reusing old problems as assignment…”
Large Language Models in Computer Science Classrooms: Ethical Challenges and Strategic Solutions
Azoulay, R., Hirst, T., & Reches, S. (2025) · Applied Sciences · doi:10.3390/app15041793 · Cited by 47
Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers
Shi, Y., Zhang, H., Wan, C., & Gu, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00005 · Cited by 67
“The two main categories of detection methods are zero-shot and training-based methods, and our DetectCodeGPT falls into the former category.”
Metadata
- Authors
- Pan, W. H., Chok, M. J., Wong, J. L. S., Shin, Y. X., Poon, Y. S., Yang, Z., Chong, C. Y., Lo, D., & Lim, M. K.
- Affiliations
- Monash University Malaysia (MY)
- Singapore Management University (SG)
- ORCID
- Venue
- Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training
- Publisher
- ACM
- Published
- 2024-04-14
- Pages
- 1-11
- OpenAlex
- W4398766125
- Cited by
- 66
- References
- 14
- Citations by year
- 2026: 26 · 2025: 29 · 2024: 11
- FWCI
- 30.10
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- IEEE/ACM International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- AI-generated code detection, large language models, ChatGPT, academic misconduct, programming education, artificial intelligence in education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Assessing AI Detectors in Identifying AI-Generated Code: Implications for Education. https://app.cochat.ai/papers/10.1145/3639474.3640068
- The paper
- Pan, W. H., Chok, M. J., Wong, J. L. S., Shin, Y. X., Poon, Y. S., Yang, Z., Chong, C. Y., Lo, D., & Lim, M. K. (2024). Assessing AI Detectors in Identifying AI-Generated Code: Implications for Education. Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training. https://doi.org/10.1145/3639474.3640068
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.