Testing the Ability of Teachers and Students to Differentiate between Essays Generated by ChatGPT and High School Students
Open accessAbstract
The release of ChatGPT in late 2022 prompted widespread concern about the implications of artificial intelligence for academic integrity, but thus far there has been little direct empirical evidence to inform this debate. Participants (69 high school teachers, 140 high school students, total N = 209 ) took an AI Identification Test in which they read pairs of essays—one written by a high school student and the other by ChatGPT—and guessed which was generated by the chatbot. Accuracy was only 70% for teachers, and it was slightly worse for students (62%). Self-reported confidence did not predict accuracy, nor did experience with ChatGPT or subject-matter expertise. Well-written student essays were especially hard to differentiate from the ChatGPT texts. In another set of measures, students reported greater optimism than their teachers did about the future role of ChatGPT in education. Students expressed disapproval of submitting ChatGPT-generated essays as one’s own but rated this and other possible academic integrity violations involving ChatGPT less negatively than teachers did. These results form an empirical basis for further work on the relationship between AI and academic integrity.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Plagiarism detection software is unreliable at best and potentially biased against students whose first language is not English, and it is becoming increasingly difficult to differentiate between human and GenAI output.
“Plagiarism detection software is unreliable at best and potentially biased against students whose fi rst language is not English (Liang et al., 2023), and with the ever-advancing sophistication of LLMs, it is becoming increasingly dif fi cult to differentiate between human and GenAI output (Waltzer et al., 2023).”
From Artificial Intelligence for Academic Purposes (AIAP): Integrating AI literacy into an EAP module (2025) · cites this paper as Waltzer et al., 2023
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483 · see citing works
Hox, J., Moerbeek, M., & van de Schoot, R. (2010) · doi:10.4324/9780203852279 · Cited by 3,469 · see citing works
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · see citing works
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079 · see citing works
The practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246 · see citing works
Artificial intelligence in education: Addressing ethical challenges in K-12 settings
Akgun, S., & Greenhow, C. (2022) · AI and Ethics · doi:10.1007/s43681-021-00096-7 · Cited by 1,473 · cited 1× · see citing works
“Another far-reaching set of questions concerns its effect on education, including its potential to promote critical thinking [8] or identify which teaching approaches work best in different contexts [9], as well as the possibility that it will exacerbate social problems such as discrimination in education [10, 11].”
BackgroundA Review of Artificial Intelligence (AI) in Education from 2010 to 2020
Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J.-B., Yuan, J., & Li, Y. (2021) · Complexity · doi:10.1155/2021/8812542 · Cited by 1,364 · cited 1× · see citing works
“Such attitudes are likely to affect how useful a role AI ends up playing in the classroom [22].”
BackgroundCelik, I. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107468 · Cited by 1,012 · see citing works
AI bot ChatGPT writes smart essays — should professors worry?
Stokel-Walker, C. (2022) · Nature · doi:10.1038/d41586-022-04397-7 · Cited by 543 · see citing works
ChatGPT: The End of Online Exam Integrity?
Susnjak, T. (2022) · arXiv · doi:10.48550/arxiv.2212.09292 · Cited by 370 · cited 2× · see citing works
“Many of these questions revolve around academic integrity in education [14, 42].”
BackgroundChocarro, R., Cortiñas, M., & Marcos-Matás, G. (2023) · Educational Studies · doi:10.1080/03055698.2020.1850426 · Cited by 343 · see citing works
The death of the short-form physics essay in the coming AI revolution
Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A., & Testrow, C. P. (2023) · Physics Education · doi:10.1088/1361-6552/acc5cf · Cited by 199 · see citing works
ChatGPT: Future Directions and Open possibilities
Aljanabi, M. (2023) · Mesopotamian Journal of CyberSecurity · doi:10.58496/mjcs/2023/003 · Cited by 143 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Influence of ChatGPT on Student Engagement: A Systematic Review and Future Research Agenda
Lo, C. K., Hew, K. F., & Jong, M. S.-Y. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105100 · Cited by 392 · see citing works
ChatGPT in Teaching and Learning: A Systematic Review
Ali, D., Fatemi, Y., Boskabadi, E., Nikfar, M., Ugwuoke, J., & Ali, H. (2024) · Education Sciences · doi:10.3390/educsci14060643 · Cited by 146 · see citing works
Artificial Intelligence for Academic Purposes (AIAP): Integrating AI literacy into an EAP module
Ngo, T. N., & Hastie, D. (2025) · English for Specific Purposes · doi:10.1016/j.esp.2024.09.001 · Cited by 72 · see citing works
“…software is unreliable at best and potentially biased against students whose fi rst language is not English (Liang et al., 2023), and with the ever-advancing sophistication of LLMs, it is becoming increasingly dif fi cult to differentiate between human and GenAI output (Waltzer et al., 2023).”
Chein, J., Martinez, S., & Barone, A. (2024) · doi:10.21203/rs.3.rs-4277893/v1 · Cited by 38 · see citing works
Zhu, Y., Zhu, C., Wu, T., Wang, S., Zhou, Y., Chen, J., Wu, F., & Li, Y. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-12898-3 · Cited by 33 · see citing works
- Authors
- Waltzer, T., Cox, R. L., & Heyman, G. D.
- Affiliations
- University of California San Diego (US)
- Corresponding author
- Heyman, Gail D.
- Venue
- Human Behavior and Emerging Technologies
- Publisher
- Wiley
- Published
- 2023-06-26
- Volume
- 2023
- Pages
- 1-9
- ISSN
- 2578-1863 (ISSN-L)
- OpenAlex
- W4382140847
- Cited by
- 61
- References
- 13
- Citations by year
- 2026: 11 · 2025: 21 · 2024: 23 · 2023: 6
- FWCI
- 15.26
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Human Behavior and Emerging Technologies (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- National Science Foundation (2104610)
- Topics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- ChatGPT, AI-generated text detection, academic integrity, student and teacher perceptions, self-reported confidence, artificial intelligence in education, high school students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
