The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Do teachers spot AI? Evaluating the detectability of AI-generated texts among student essays
Open accessAbstract
The potential application of generative artificial intelligence (AI) in schools and universities poses great challenges, especially for the assessment of students’ texts. Previous research has shown that people generally have difficulty distinguishing AI-generated from human-written texts; however, the ability of teachers to identify an AI-generated text among student essays has not yet been investigated. Here we show in two experimental studies that novice (N = 89) and experienced teachers (N = 200) could not identify texts generated by ChatGPT among student-written texts. However, there are some indications that more experienced teachers made more differentiated and more accurate judgments. Furthermore, both groups were overconfident in their judgments. Effects of real and assumed source on quality assessment were heterogeneous. Our findings demonstrate that with relatively little prompting, current AI can generate texts that are not detectable for teachers, which poses a challenge to schools and universities in grading student essays. Our study provides empirical evidence for the current debate regarding exam strategies in schools and universities in light of the latest technological developments.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous studies suggested that human users often fail to recognise AI-generated text and are generally overconfident in their ability to do so (Casal & Kessler, 2023;Fleckenstein et al., 2024).”
From “ ChatGPT can make mistakes. Check important info.” Epistemic beliefs and metacognitive accuracy in students' integration of ChatGPT content into academic writing (2025) · cites this paper as Fleckenstein et al., 2024
In the citing paper
“This is heightened by the fact that even experts, such as professional linguists and teachers, are unable to effectively distinguish between human-written and AI-generated text (Casal & Kessler, 2023;Fleckenstein et al., 2024).”
From “ ChatGPT can make mistakes. Check important info.” Epistemic beliefs and metacognitive accuracy in students' integration of ChatGPT content into academic writing (2025) · cites this paper as Fleckenstein et al., 2024
Citations
Cites
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022) · doi:10.1101/2022.12.23.521610 · Cited by 1,099
Köbis, N., & Mossink, L. D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106553 · Cited by 406
Readers’ perception of computer-generated news: Credibility, expertise, and readability
Graefe, A., Haim, M., Haarmann, B., & Brosius, H.-B. (2018) · Journalism · doi:10.1177/1464884916641269 · Cited by 315
Meyer, J., Jansen, T., Schiller, R., Liebenow, L. W., Steinbach, M., Horbach, A., & Fleckenstein, J. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100199 · Cited by 311
Self-citationSailer, M., Bauer, E., Hofmann, R., Kiesewetter, J., Glas, J., Gurevych, I., & Fischer, F. (2023) · Learning and Instruction · doi:10.1016/j.learninstruc.2022.101620 · Cited by 106
Keller, S., Fleckenstein, J., Krüger, M., Köller, O., & Rupp, A. A. (2020) · Journal of Second Language Writing · doi:10.1016/j.jslw.2019.100700 · Cited by 65
Self-citationFleckenstein, J., Keller, S., Krüger, M., Tannenbaum, R. J., & Köller, O. (2019) · Assessing Writing · doi:10.1016/j.asw.2019.100420 · Cited by 59
Self-citationAutomated Essay Scoring at Scale: A Case Study in Switzerland and Germany
Rupp, A., Casabianca, J., Krüger, M., Keller, S., & Köller, O. (2019) · ETS Research Report Series · doi:10.1002/ets2.12249 · Cited by 39
Self-citationJansen, T., Vögelin, C., Machts, N., Keller, S., Köller, O., & Møller, J. (2020) · Teaching and Teacher Education · doi:10.1016/j.tate.2020.103216 · Cited by 37
Self-citationGunser, V. E., Gottschling, S., Brucker, B., Richter, S., Çakir, D., & Gerjets, P. (2022) · IN2WRITING · doi:10.18653/v1/2022.in2writing-1.8 · Cited by 30
Jansen, T., Meyer, J., Fleckenstein, J., Horbach, A., Keller, S., & Møller, J. (2023) · Learning and Instruction · doi:10.1016/j.learninstruc.2023.101847 · Cited by 28
Self-citationJudgment accuracy of German student texts: Do teacher experience and content knowledge matter?
Møller, J., Jansen, T., Fleckenstein, J., Machts, N., Meyer, J., & Reble, R. (2022) · Teaching and Teacher Education · doi:10.1016/j.tate.2022.103879 · Cited by 21
Self-citationEmpathic AI Painter: A Computational Creativity System with Embodied Conversational InteractionPreprint
Yalçın, Ö. N., Abukhodair, N., & DiPaola, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.14223 · Cited by 9
Cited by
Sorted by citations
Human-AI agency in the age of generative AI
Krakowski, S. M. (2025) · Information and Organization · doi:10.1016/j.infoandorg.2025.100560 · Cited by 223
Weng, X., XIA, Q., Gu, M., Rajaram, K., & Chiu, T. K. F. (2024) · Australasian Journal of Educational Technology · doi:10.14742/ajet.9540 · Cited by 161
“…thinking, problem-solving and communication skills (Almasre, 2024; Banihashem et al., 2024; Bearman et al., 2024; Blanke, 2024; Cotton et al., 2023; Fleckenstein et al., 2024); and open-ended assessments in which students were encouraged in aspects of originality and creativity (Eager & Brunton,…”
BackgroundInfluentialSimple techniques to bypass GenAI text detectors: implications for inclusive education
Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., McGaughran, J., & Khuat, H. Q. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00487-w · Cited by 117
“This model represented a step-change in the abilities of GenAI to create text that was significantly more difficult to detect by educators (Fleckenstein et al., 2024; Perkins et al., 2023), and thus the technology began to draw greater attention in the public sphere, as well as in educational…”
Acosta-Enriquez, B. G., Várgas, C. G. A. P., Jordan, O. H., Ballesteros, M. A. A., & Morales, A. E. P. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100320 · Cited by 99
Mizumoto, A., Yasuda, S., & Tamura, Y. (2024) · Applied Corpus Linguistics · doi:10.1016/j.acorp.2024.100106 · Cited by 58
Metadata
- Authors
- Fleckenstein, J., Meyer, J., Jansen, T., Keller, S. D., Köller, O., & Möller, J.
- Affiliations
- University of Hildesheim (DE)
- Leibniz Institute for Science and Mathematics Education (DE)
- Zurich University of Teacher Education (CH)
- Corresponding author
- Fleckenstein, Johanna
- Venue
- Computers and Education: Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 2024-06
- Volume
- 6
- Pages
- 100209
- ISSN
- 2666-920X (ISSN-L)
- OpenAlex
- W4391222332
- Cited by
- 126
- References
- 16
- Citations by year
- 2026: 58 · 2025: 48 · 2024: 20
- FWCI
- 47.38
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Computers and Education Artificial Intelligence (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Education › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- AI-generated text detection, generative AI, ChatGPT, student essays, teacher judgment, essay scoring, overconfidence, source effects, academic integrity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
