Cited by 800Artificial Intelligence in EducationEFL/ESL Teaching and LearningStudent Assessment and Feedback

AI-generated feedback on writing: insights into efficacy and ENL student preference

Open access
Escalante, Juan, Pack, Austin, Barrett, Alex·2023·International Journal of Educational Technology in Higher Education

Abstract

The question of how generative AI tools, such as large language models and chatbots, can be leveraged ethically and effectively in education is ongoing. Given the critical role that writing plays in learning and assessment within educational institutions, it is of growing importance for educators to make thoughtful and informed decisions as to how and in what capacity generative AI tools should be leveraged to assist in the development of students’ writing skills. This paper reports on two longitudinal studies. Study 1 examined learning outcomes of 48 university English as a new language (ENL) learners in a six-week long repeated measures quasi experimental design where the experimental group received writing feedback generated from ChatGPT (GPT-4) and the control group received feedback from their human tutor. Study 2 analyzed the perceptions of a different group of 43 ENLs who received feedback from both ChatGPT and their tutor. Results of study 1 showed no difference in learning outcomes between the two groups. Study 2 results revealed a near even split in preference for AI-generated or human-generated feedback, with clear advantages to both forms of feedback apparent from the data. The main implication of these studies is that the use of AI-generated feedback can likely be incorporated into ENL essay evaluation without affecting learning outcomes, although we recommend a blended approach that utilizes the strengths of both forms of feedback. The main contribution of this paper is in addressing generative AI as an automatic essay evaluator while incorporating learner perspectives.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s41239-023-00425-2

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Students considered AI-generated feedback more readable and detailed compared to human-generated feedback.

    “This explanation is supported by a study from Escalante, Pack, and Barrett (2023) in which students considered AI-generated feedback more 'readable and detailed' compared to human-generated feedback.”

    From Summarise, elaborate, try again: exploring the effect of feedback literacy on AI-enhanced essay writing (2026) · cites this paper as Escalante, Pack, and Barrett (2023)

    In Study 2 (Relating to RQ2), students who preferred AI feedback cited its clarity, ease of understanding, and detailed comments; one contrasted its extensive feedback with tutors’ limited time. But the Discussion and conclusion attributes the exact finding that AI feedback was more “readable and detailed” than instructor feedback to Dai et al. (2023), while this study found preferences were roughly evenly split. The citing sentence overstates that comparison as a finding about Escalante et al.’s students generally. (LLM, conf=0.85)

    Experimental — accuracy not yet measured
  • Combining AI-driven feedback with human input may provide comprehensive guidance tailored to individual needs.

    “Combining AI-driven feedback with human input may provide comprehensive guidance tailored to individual needs [142].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Suleyman, M. (2023). My new Turing test would see if AI can make $1 million. MIT Technology Review. https:// www. techn ology review. com/ 2023/ 07/ 14/ 10762 96/ musta fa-suley man-my-new-turing-test-would-see-if-ai-can-make-1-milli on/

Cited by

Sorted by citations

Authors
Escalante, J., Pack, A., & Barrett, A.
Affiliations
  • Brigham Young University–Hawaii (US)
  • Florida State University (US)
Corresponding author
Escalante, Juan
Venue
International Journal of Educational Technology in Higher Education
Published
2023-10-27
Volume
20
Issue
1
ISSN
2365-9440 (ISSN-L)
OpenAlex
W4387954002
Cited by
589
References
38
Citations by year
2026: 267 · 2025: 231 · 2024: 90 · 2023: 1
FWCI
138.70
Type
journal article
Language
en
Open access
Locations
  • International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Social Sciences › Education › Student Assessment and Feedback
Keywords
AI-generated feedback, generative AI in education, ChatGPT, large language models, writing instruction, learner preferences, writing learning outcomes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review