The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
AI-generated feedback on writing: insights into efficacy and ENL student preference
Open accessAbstract
The question of how generative AI tools, such as large language models and chatbots, can be leveraged ethically and effectively in education is ongoing. Given the critical role that writing plays in learning and assessment within educational institutions, it is of growing importance for educators to make thoughtful and informed decisions as to how and in what capacity generative AI tools should be leveraged to assist in the development of students’ writing skills. This paper reports on two longitudinal studies. Study 1 examined learning outcomes of 48 university English as a new language (ENL) learners in a six-week long repeated measures quasi experimental design where the experimental group received writing feedback generated from ChatGPT (GPT-4) and the control group received feedback from their human tutor. Study 2 analyzed the perceptions of a different group of 43 ENLs who received feedback from both ChatGPT and their tutor. Results of study 1 showed no difference in learning outcomes between the two groups. Study 2 results revealed a near even split in preference for AI-generated or human-generated feedback, with clear advantages to both forms of feedback apparent from the data. The main implication of these studies is that the use of AI-generated feedback can likely be incorporated into ENL essay evaluation without affecting learning outcomes, although we recommend a blended approach that utilizes the strengths of both forms of feedback. The main contribution of this paper is in addressing generative AI as an automatic essay evaluator while incorporating learner perspectives.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This explanation is supported by a study from Escalante, Pack, and Barrett (2023) in which students considered AI-generated feedback more 'readable and detailed' compared to human-generated feedback.”
From Summarise, elaborate, try again: exploring the effect of feedback literacy on AI-enhanced essay writing (2026) · cites this paper as Escalante, Pack, and Barrett (2023)
Experimental — accuracy not yet measuredIn the citing paper
“Combining AI-driven feedback with human input may provide comprehensive guidance tailored to individual needs [142].”
From Leveraging AI in E-Learning: Personalized Learning and Adaptive Assessment through Cognitive Neuropsychology—A Systematic Analysis (2024) · cites this paper as [142]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
Principles and Practice in Second Language Acquisition
Higgs, T. V., & Krashen, S. D. (1983) · Modern Language Journal · doi:10.2307/328293 · Cited by 5,375
Principles and Practice in Second Language Acquisition
Krahnke, K. J., & Krashen, S. D. (1983) · TESOL Quarterly · doi:10.2307/3586656 · Cited by 4,483
Language Learning with Technology
Castillo, D. G. (2021) · Springer eBooks · doi:10.1007/978-981-16-2697-5 · Cited by 2,211
Clinical versus mechanical prediction: a meta-analysis
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.1.19 · Cited by 1,640
Sociocultural approaches to learning and development: A Vygotskian framework
John-Steiner, V. P., & Mahn, H. (1996) · Educational Psychologist · doi:10.1080/00461520.1996.9653266 · Cited by 1,389
A SWOT analysis of ChatGPT: Implications for educational practice and research
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2195846 · Cited by 1,384
Chiu, T. K. F., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100118 · Cited by 1,181 · cited 2×
“Personalized learning through evaluating and giving feedback on essay writing has been identified as a potential strength of GenAI (Chiu et al., 2023; Farrokhnia et al., 2023; Zhu et al., 2023), which can, in turn, help decrease teacher workload (Farrokhnia et al., 2023) and prevent teacher burnout.”
BackgroundChatGPT in higher education: considerations for academic integrity and student learning
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.17 · Cited by 853 · cited 1×
“…across Australia, New Zealand, The U.K., and the U.S., general sentiment appeared evenly split between positive and negative, but concerns about academic integrity have been raised (Sullivan, 2023), with some educational institutions deciding to ban ChatGPT than to allow its use (Yang, 2023).”
BackgroundExploring the potential of using an AI language model for automated essay scoring
Mizumoto, A., & Eguchi, M. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100050 · Cited by 647
Handbook of Automated Essay Evaluation
(2013) · Routledge eBooks · doi:10.4324/9780203122761 · Cited by 518
Koltovskaia, S. (2020) · Assessing Writing · doi:10.1016/j.asw.2020.100450 · Cited by 447
Can Large Language Models Provide Feedback to Students? A Case Study on ChatGPT
Dai, W., Lin, J., Jin, F., Li, T., Tsai, Y.-S., Gasevic, D., & Chen, G. (2023) · doi:10.35542/osf.io/hcgzj · Cited by 395
Can Large Language Models Provide Feedback to Students? A Case Study on ChatGPT
Dai, W., Lin, J., Jin, H., Li, T., Tsai, Y.-S., Gašević, D., & Chen, G. (2023) · 2023 IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt58122.2023.00100 · Cited by 395 · cited 3×
“This echoes the findings of Dai et al. (2023), namely that AI-generated feedback was found to be more readable and detailed than feedback from an instructor.”
BackgroundMethodInfluentialA large-scale comparison of human-written versus ChatGPT-generated essays
Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z., & Trautsch, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-45644-9 · Cited by 338
Large Language Models Are Human-Level Prompt Engineers
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., & Ba, J. (2022) · arXiv · doi:10.48550/arxiv.2211.01910 · Cited by 330
Partnering with AI: Intelligent writing assistance and instructed language learning
Godwin-Jones, R. (2022) · Language Learning & Technology · doi:10.64152/10125/73474 · Cited by 285
ChatGPT in Education: Global Reactions to AI Innovations
(2023) · doi:10.21203/rs.3.rs-2840105/v1 · Cited by 242
ChatGPT in education: global reactions to AI innovations
Fütterer, T., Fischer, C., Alekseeva, A., Chen, X., Tate, T., Warschauer, M., & Gerjets, P. (2023) · Scientific Reports · doi:10.1038/s41598-023-42227-6 · Cited by 242 · cited 2×
“The prevailing sentiment on GenAI seems to be that reforms are needed to adapt educational practices in accommodation of the technology (Fütterer et al., 2023; Tseng & Warschauer, 2023).”
BackgroundGrammarly as AI-powered English Writing Assistant: Students' Alternative for Writing English
Fitria, T. N. (2021) · Metathesis: Journal of English Language, Literature, and Teaching · doi:10.31002/metathesis.v5i1.3519 · Cited by 221
How to harness the potential of ChatGPT in education?
Zhu, C., Sun, M., Luo, J., Li, T., Wang, M., & |, |. (2023) · Knowledge Management & E-Learning An International Journal · doi:10.34105/j.kmel.2023.15.008 · Cited by 219
Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation With Large Language Models
Strobelt, H., Webson, A., Sanh, V., Hoover, B., Beyer, J., Pfister, H., & Rush, A. M. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209479 · Cited by 204
Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models
Strobelt, H., Webson, A., Sanh, V., Hoover, B., Beyer, J., Pfister, H., & Rush, A. M. (2022) · arXiv · doi:10.48550/arxiv.2208.07852 · Cited by 204
Can GPT-3 pass a writer’s turing test?
Elkins, K., & Chun, J. (2020) · Journal of Cultural Analytics · doi:10.22148/001c.17212 · Cited by 202
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023) · arXiv · doi:10.48550/arxiv.2303.13408 · Cited by 150 · cited 1×
“Educators have begun to use AI-powered plagiarism detectors to identify student submissions that were generated by AI, yet AI paraphrasing programs like Quillbot have been found to render AI-generated text indetectable by such tools (Krishna et al., 2023).”
BackgroundParaphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1195 · Cited by 150
Academic integrity in the age of Artificial Intelligence (AI) authoring apps
Yeo, M. A. (2023) · TESOL Journal · doi:10.1002/tesj.716 · Cited by 142
Educational Research and AI-Generated Writing: Confronting the Coming Tsunami
Tate, T. P., Doroudi, S., Ritchie, D., Xu, Y., & uci, M. W. (2023) · doi:10.35542/osf.io/4mec3 · Cited by 137
Beyond the Safeguards: Exploring the Security Risks of ChatGPTPreprint
Derner, E., & Batistič, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.08005 · Cited by 135
Behizadeh, N., & Engelhard, G. (2011) · Assessing Writing · doi:10.1016/j.asw.2011.03.001 · Cited by 135
AI-writing tools in education: If you can’t beat them, join them
Tseng, W., & Warschauer, M. (2023) · Journal of China Computer-Assisted Language Learning · doi:10.1515/jccall-2023-0008 · Cited by 119 · cited 1×
“The prevailing sentiment on GenAI seems to be that reforms are needed to adapt educational practices in accommodation of the technology (Fütterer et al., 2023; Tseng & Warschauer, 2023).”
BackgroundA systematic review of automated writing evaluation systems
Shi, H., & Aryadoust, V. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11200-7 · Cited by 111
Roscoe, R. D., Wilson, J., Johnson, A. C., & Mayra, C. R. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2016.12.076 · Cited by 98
Faculty members’ use of artificial intelligence to grade student papers: a case of implications
Kumar, R. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00130-7 · Cited by 78
Leveraging AI tools to develop the writer rather than the writing
Ingley, S. J., & Pack, A. (2023) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2023.05.007 · Cited by 65
Self-citationAssisting academics to identify computer generated writing
Abd-Elaal, E.-S., Gamage, S. H. P. W., & Mills, J. E. (2022) · European Journal of Engineering Education · doi:10.1080/03043797.2022.2046709 · Cited by 62
The Functions of Writing in an Elementary Classroom
Florio, S., & Clark, C. M. (1982) · Research in the Teaching of English · doi:10.58680/rte198215743 · Cited by 41
English as a Second Language Writing and Automated Essay Evaluation
Weigle, S. C. (2015) · Routledge eBooks · doi:10.4324/9780203122761.ch3 · Cited by 37
Urlaub, P., & Dessein, E. (2022) · L2 Journal · doi:10.5070/l214151790 · Cited by 33
Lampropoulos, G., Ferdig, R. E., & Kaplan-Rakowski, R. (2023) · doi:10.2139/ssrn.4468181 · Cited by 33
AI, write an essay for me: A large-scale comparison of human-written versus ChatGPT-generated essays
Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z., & Trautsch, A. (2023) · arXiv · Cited by 18
Gpt-4: A Review on Advancements and Opportunities in Natural Language ProcessingPreprint
Baktash, J. A., & Dawodi, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.03195 · Cited by 17
Automated Essay Evaluation and the Teaching of Writing
Elliot, N., & Klobucar, A. (2015) · Routledge eBooks · doi:10.4324/9780203122761.ch2 · Cited by 10
Cited by
Sorted by citations
The promise and challenges of generative AI in education
Giannakos, M., Azevedo, R., Brusilovsky, P., Cukurova, M., Dimitriadis, Y., Hernandez-Leo, D., Järvelä, S., Mavrikis, M., & Rienties, B. (2025) · Behaviour & Information Technology · doi:10.1080/0144929x.2024.2394886 · Cited by 543
The Influence of ChatGPT on Student Engagement: A Systematic Review and Future Research Agenda
Lo, C. K., Hew, K. F., & Jong, M. S.-Y. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105100 · Cited by 392
Prompt engineering in higher education: a systematic review to help inform curricula
Lee, D., & Palmer, E. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00503-7 · Cited by 388
Halkiopoulos, C., & Gkintoni, E. (2024) · Electronics · doi:10.3390/electronics13183762 · Cited by 328
“Combining AI-driven feedback with human input may provide comprehensive guidance tailored to individual needs [142].”
Wang, J., & Fan, W. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04787-y · Cited by 322
Retracted
Metadata
- Authors
- Escalante, J., Pack, A., & Barrett, A.
- Affiliations
- Brigham Young University–Hawaii (US)
- Florida State University (US)
- Corresponding author
- Escalante, Juan
- Venue
- International Journal of Educational Technology in Higher Education
- Published
- 2023-10-27
- Volume
- 20
- Issue
- 1
- ISSN
- 2365-9440 (ISSN-L)
- OpenAlex
- W4387954002
- Cited by
- 589
- References
- 38
- Citations by year
- 2026: 267 · 2025: 231 · 2024: 90 · 2023: 1
- FWCI
- 138.70
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Education › Student Assessment and Feedback
- Keywords
- AI-generated feedback, generative AI in education, ChatGPT, large language models, writing instruction, learner preferences, writing learning outcomes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
