The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
ChatGPT versus human essayists: an exploration of the impact of artificial intelligence for authorship and academic integrity in the humanities
Open accessAbstract
Generative AI has prompted educators to reevaluate traditional teaching and assessment methods. This study examines AI’s ability to write essays analysing Old English poetry; human markers assessed and attempted to distinguish them from authentic analyses of poetry by first-year undergraduate students in English at the University of Oxford. Using the standard UK University grading system, AI-written essays averaged a score of 60.46, whilst human essays achieved 63.57, a margin of difference not statistically significant ( p = 0.10). Notably, student submissions applied a nuanced understanding of cultural context and secondary criticism to their close reading, while AI essays often described rather than analysed, lacking depth in the evaluation of poetic features, and sometimes failing to properly recognise key aspects of passages. Distinguishing features of human essays included detailed and sustained analysis of poetic style, as well as spelling errors and lack of structural cohesion. AI essays, on the other hand, exhibited a more formal structure and tone but sometimes fell short in incisive critique of poetic form and effect. Human markers correctly identified the origin of essays 79.41% of the time. Additionally, we compare three purported AI detectors, finding that the best, ‘Quillbot’, correctly identified the origin of essays 95.59% of the time. However, given the high threshold for academic misconduct, conclusively determining origin remains challenging. The research also highlights the potential benefits of generative AI’s ability to advise on structuring essays and suggesting avenues for research. We advocate for transparency regarding AI’s capabilities and limitations, and this study underscores the importance of human critical engagement in teaching and learning in Higher Education. As AI’s proficiency grows, educators must reevaluate what authentic assessment is, and consider implementing dynamic, holistic methods to ensure academic integrity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The findings revealed that AI essays achieved an average score of 60.46, closely trailing human submissions, which averaged 63.57 a difference that is not statistically significant (p = 0.10) [6].”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many institutions are moving toward more authentic and formative assessment practices.These include in-class written tasks, oral presentations, staged writing processes with feedback loops, and portfolio-based evaluations [6].”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AI-generated essays in humanities disciplines often meet acceptable grading criteria, mimicking the surface-level features of human writing [6].”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While lacking critical analysis, these essays are well-organized, grammatically polished, and difficult to differentiate from human-written content without detailed forensic examination.”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [6]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, the study raised concerns about the fairness of detection algorithms, highlighting the risk that non-native English speakers could be disproportionately penalized due to linguistic patterns that diverge from native norms [6].”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [6]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 1×
“There is promising potential in using ChatGPT as a research and learning assistant (Kas-neci et al. 2023).”
BackgroundZawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242 · cited 3×
“The urgency for comprehensive insights into AI’s ramifications for both teaching and assessment has only intensified in an era where powerful generative AI tools are publicly-accessible (Zawacki-Richter et al. 2019).”
BackgroundArtificial Intelligence in Education: A Review
Chen, L., Chen, P., & Lin, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2988510 · Cited by 4,133 · cited 1×
“Numerous review articles portrayed artificial intelligence as a significant asset to academic administration (Chen et al. 2020) and a pathway to more dynamic learning experiences.”
BackgroundArtificial intelligence in higher education: the state of the field
Crompton, H., & Burke, D. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00392-8 · Cited by 2,010 · cited 2×
“Like many STEM subjects, language acquisition can be assessed more objectively, and a relatively linear progression of skills and complexity can be arranged.”
BackgroundThe promises and challenges of artificial intelligence for teachers: A systematic review of research
Celik, I., Dindar, M., Muukkonen, H., & Järvelä, S. (2022) · TechTrends · doi:10.1007/s11528-022-00715-y · Cited by 984 · cited 1×
“Intriguingly, challenges highlighted in AI-focused literature as recent as 2022 included the ‘[l]imited technical capacity of AI’ and the ‘[i]napplicability of the AI system to multiple settings’ (Celik et al. 2022), rather than the evident risks to assessment authenticity observed today.”
BackgroundArtificial intelligence in education : challenges and opportunities for sustainable development
Pedró, F., Subosa, M., Rivas, A., & Valverde, P. (2019) · MINISTERIO DE EDUCACIÓN · Cited by 902
Recipes for Building an Open-Domain Chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., et al. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.24 · Cited by 784
Recipes for building an open-domain chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Shuster, K., Smith, E. M., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.13637 · Cited by 784 · cited 1×
“While recommending relevant journals, an essay by J.R.R. Tolkien, and Seamus Heaney’s adaptation of “Beowulf”, the other references were either mis-attributed to authors who have produced similar works, or entirely “hallucinated” (Roller et al. 2021), in each case with an inaccurate summary.”
MethodGPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · Patterns · doi:10.1016/j.patter.2023.100779 · Cited by 673 · cited 2×
“Beyond these challenges, other studies have highlighted textual nuances, such as the presence or absence of specific phrases and symbols, as potential markers for distinguishing between human and AI-generated content (Desaire et al. 2023).”
Background
Research works printed but not linked: 6
- Baird JA, Greatorex J, Bell JF (2004) What makes marking reliable? experiments with uk examinations. Assess Educ Princ Policy Pract 11(3):331–348
- Gupta S, Chen Y (2022) Supporting inclusive learning using chatbots? a chatbot-led interview study. J Inf Syst Educ 33(1):98–108
- Hu X, Chen PY, Ho TY (2023) Radar: Robust ai-text detection via adversarial learning. Adv Neural Inf Process Syst 36:15077–15095
- Jannai D, Meron A, Lenz B, Levine Y, Shoham Y (2023) Human or not? a gamified approach to the turing test. arXiv preprint arXiv:2305.20010 · arXiv
- Liu Z, Yao Z, Li F, Luo B (2023) Check me if you can: Detecting chatgpt-generated academic writing using checkgpt. arXiv preprint arXiv:2306.05524 · arXiv
- Sadasivan VS, Kumar A, Balasubramanian S, Wang W, Feizi S (2023) Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156 · arXiv
Cited by
Tejeda, A. S., Oller, J. C. D., Escandell, J. M. B., Gómez-Díaz, R., & Rodríguez, A. G. (2026) · Frontiers in Education · doi:10.3389/feduc.2025.1711718 · Cited by 130
“Some of these tools have been criticized for “hallucinating” content or providing incorrect information, a phenomenon that refers to the generation of inaccurate responses that nevertheless appear realistic (Kim et al., 2025; Revell et al., 2024).”
InfluentialKrupp, L., Steinert, S., Kiefer-Emmanouilidis, M., Avila, K. E., Lukowicz, P., Kuhn, J., Küchemann, S., & Karolus, J. (2024) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia240195 · Cited by 82
Does ChatGPT Write Like a Student? Engagement Markers in Argumentative Essays
Jiang, F. (., & Hyland, K. (2025) · Written Communication · doi:10.1177/07410883251328311 · Cited by 52
“Others express concern that “the eerily humanlike chatbot” (Satariano & Kang, 2023) might make it difficult to distinguish GPT-generated and human-authored texts in assessing students’ writing (Revell et al., 2024).”
Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education
Deep, P. D., Edgington, W. D., Ghosh, N., & Rahaman, M. S. (2025) · Information · doi:10.3390/info16100905 · Cited by 26
Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education
Deep, P. D., Edgington, W. D., Ghosh, N., & Rahaman, M. S. (2025) · doi:10.20944/preprints202507.2233.v1 · Cited by 26
Metadata
- Authors
- Revell, T., Yeadon, W., Cahilly-Bretzin, G., Clarke, I., Manning, G., Jones, J., Mulley, C., Pascual, R. J., Bradley, N., Thomas, D., & Leneghan, F.
- Affiliations
- University of Oxford (GB)
- Durham University (GB)
- University of Liverpool (GB)
- Corresponding author
- Revell, T.
- Venue
- International Journal for Educational Integrity
- Published
- 2024-10-21
- Volume
- 20
- Issue
- 1
- ISSN
- 1833-2595 (ISSN-L)
- OpenAlex
- W4403579585
- Cited by
- 51
- References
- 29
- Citations by year
- 2026: 33 · 2025: 17 · 2024: 1
- FWCI
- 21.14
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- International Journal for Educational Integrity (publishedVersion, cc-by)
- Durham Research Online (Durham University) (publishedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- UK Research and Innovation
- Arts and Humanities Research Council
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- generative AI, ChatGPT, academic integrity, essay scoring, AI detection, Old English poetry, close reading, essay marking, QuillBot, artificial intelligence in higher education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatGPT versus human essayists: an exploration of the impact of artificial intelligence for authorship and academic integrity in the humanities. https://app.cochat.ai/papers/10.1007/s40979-024-00161-8
- The paper
- Revell, T., Yeadon, W., Cahilly-Bretzin, G., Clarke, I., Manning, G., Jones, J., Mulley, C., Pascual, R. J., Bradley, N., Thomas, D., & Leneghan, F. (2024). ChatGPT versus human essayists: an exploration of the impact of artificial intelligence for authorship and academic integrity in the humanities. International Journal for Educational Integrity. https://doi.org/10.1007/s40979-024-00161-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.