Cited by 56Artificial Intelligence in EducationEFL/ESL Teaching and LearningStudent Assessment and Feedback

Enhancing university level English proficiency with generative AI: empirical insights into automated feedback and learning outcomes

Open access
Chan, Sumie Tsz Sum, Lo, Noble Po Kan, Wong, Alan Man Him·2024·Contemporary Educational Technology

Abstract

This paper investigates the effects of large language model (LLM) based feedback on the essay writing proficiency of university students in Hong Kong. It focuses on exploring the potential improvements that generative artificial intelligence (AI) can bring to student essay revisions, its effect on student engagement with writing tasks, and the emotions students experience while undergoing the process of revising written work. Utilizing a randomized controlled trial, it draws comparisons between the experiences and performance of 918 language students at a Hong Kong university, some of whom received generated feedback (GPT-3.5-turbo LLM) and some of whom did not. The impact of AI-generated feedback is assessed not only through quantifiable metrics, entailing statistical analysis of the impact of AI feedback on essay grading, but also through subjective indices, student surveys that captured motivational levels and emotional states, as well as thematic analysis of interviews with participating students. The incorporation of AI-generated feedback into the revision process demonstrated significant improvements in the caliber of students’ essays. The quantitative data suggests notable effect sizes of statistical significance, while qualitative feedback from students highlights increases in engagement and motivation as well as a mixed emotional experience during revision among those who received AI feedback.

Integrity signals

Registry

Crossref · 10.30935/cedtech/15607

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

The text we found for this paper looks like another document, so its claims and self-check weren’t run. The registry and citation checks still apply.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The text we found for this paper looks like another document, so the self-check wasn’t run.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. McCormick, K. (2015). SPSS statistics for dummies. John Wiley.
  2. Salcedo, J., & McCormick, K. (2020). SPSS statistics (4th ed.). John Wiley.
  3. Chan, Sumie Tsz Sum, Noble Po Kan Lo, and Alan Man Him Wong. "Enhancing university level English proficiency with generative AI: Empirical insights into automated feedback and learning outcomes". Contemporary Educational Technology 2024 16 no. 4 (2024): ep541. https://doi.org/10.30935/cedtech/15607 Chan, Sumie Tsz Sum et al. "Enhancing university level English proficiency with generative AI: Empirical insights into automated feedback and learning outcomes". Contemporary Educational Technology, vol. 16, no. 4, 2024, ep541. https://doi.org/10.30935/cedtech/15607 Chan STS, Lo NPK, Wong AMH. Enhancing university level English proficiency with generative AI: Empirical insights into automated feedback and learning outcomes. CONT ED TECHNOLOGY. 2024;16(4):ep541. https://doi.org/10.30935/cedtech/15607

Cited by

Sorted by citations

Authors
Chan, S. T. S., Lo, N. P. K., & Wong, A. M. H.
Affiliations
  • Chinese University of Hong Kong (HK)
  • Hong Kong Polytechnic University (HK)
Corresponding author
Lo, Noble Po Kan
Venue
Contemporary Educational Technology
Published
2024-11-13
Volume
16
Issue
4
Pages
ep541
ISSN
1309-517X (ISSN-L)
OpenAlex
W4404332403
Cited by
56
References
52
Citations by year
2026: 35 · 2025: 21
FWCI
21.00
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Contemporary Educational Technology (publishedVersion, cc-by)
  • PolyU Institutional Research Archive (Hong Kong Polytechnic University) (publishedVersion, cc-by)
  • Lancaster EPrints (Lancaster University) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Social Sciences › Education › Student Assessment and Feedback
Keywords
generative AI, large language models, AI-generated feedback, essay writing, English proficiency, university students, essay revision, student motivation, student engagement, emotions in writing, GPT-3.5-turbo, writing instruction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review