CoChat
Sign Up
Cited by 800Artificial Intelligence in EducationEFL/ESL Teaching and LearningStudent Assessment and Feedback

The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.

AI-generated feedback on writing: insights into efficacy and ENL student preference

Open access
Escalante, Juan, Pack, Austin, Barrett, Alex·2023·International Journal of Educational Technology in Higher Education
DOI 10.1186/s41239-023-00425-2
Open this paper in CoChat — free accountOpen PDF

Abstract

The question of how generative AI tools, such as large language models and chatbots, can be leveraged ethically and effectively in education is ongoing. Given the critical role that writing plays in learning and assessment within educational institutions, it is of growing importance for educators to make thoughtful and informed decisions as to how and in what capacity generative AI tools should be leveraged to assist in the development of students’ writing skills. This paper reports on two longitudinal studies. Study 1 examined learning outcomes of 48 university English as a new language (ENL) learners in a six-week long repeated measures quasi experimental design where the experimental group received writing feedback generated from ChatGPT (GPT-4) and the control group received feedback from their human tutor. Study 2 analyzed the perceptions of a different group of 43 ENLs who received feedback from both ChatGPT and their tutor. Results of study 1 showed no difference in learning outcomes between the two groups. Study 2 results revealed a near even split in preference for AI-generated or human-generated feedback, with clear advantages to both forms of feedback apparent from the data. The main implication of these studies is that the use of AI-generated feedback can likely be incorporated into ENL essay evaluation without affecting learning outcomes, although we recommend a blended approach that utilizes the strengths of both forms of feedback. The main contribution of this paper is in addressing generative AI as an automatic essay evaluator while incorporating learner perspectives.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s41239-023-00425-2

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Mean responses to item 9, where 1 equals preference for human feedback and 2 equals preference for AI feedback
Fig. 1 Mean responses to item 9, where 1 equals preference for human feedback and 2 equals preference for AI feedbackCC-BY · From the PDF
Fig. 2 Means of items 1-8
Fig. 2 Means of items 1-8CC-BY · From the PDF
Image 3
CC-BY · From the PDF
Fig. 4 Means of items 3 and 4 relating to clarity
Fig. 4 Means of items 3 and 4 relating to clarityCC-BY · From the PDF
Fig. 5 Means of items 5 and 6 relating to helpfulness
Fig. 5 Means of items 5 and 6 relating to helpfulnessCC-BY · From the PDF
Fig. 6 Means of items 7 and 8 relating to preference
Fig. 6 Means of items 7 and 8 relating to preferenceCC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 38 references

Cited by

Sorted by citations

See all 800

Related

Metadata

Authors
Escalante, J., Pack, A., & Barrett, A.
Affiliations
  • Brigham Young University–Hawaii (US)
  • Florida State University (US)
Corresponding author
Escalante, Juan
ORCID
Venue
International Journal of Educational Technology in Higher Education
Publisher
Springer Science and Business Media LLC
Published
2023-10-27
Volume
20
Issue
1
ISSN
2365-9440 (ISSN-L)
DOI
10.1186/s41239-023-00425-2
OpenAlex
W4387954002
Cited by
589
References
38
Citations by year
2026: 267 · 2025: 231 · 2024: 90 · 2023: 1
FWCI
138.70
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Social Sciences › Education › Student Assessment and Feedback
Keywords
AI-generated feedback, generative AI in education, ChatGPT, large language models, writing instruction, learner preferences, writing learning outcomes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review