CoChat

Generative AI as an Automated Written Corrective Feedback Provider in EFL Academic Writing: A Mixed-Methods Investigation of Accuracy Gains, Feedback Quality, and Learner Engagement

Open access
Mujtaba Kamal Pasha · 2026 · Journal of Artificial Intelligence and Computing

The DOI matches its Crossref record; 20 of the 22 references it cites were found in a registry or an index; 4 of the 46 sources its claims cite were checked for support.

Abstract

The integration of generative artificial intelligence (GenAI) and large language models (LLMs) into language technology has reopened long-standing questions about the role of automated written corrective feedback (AWCF) in second-language (L2) writing development. While early enthusiasm has positioned conversational LLMs such as GPT-4 as scalable substitutes for the labour-intensive provision of teacher feedback, the empirical base remains fragmented, and few controlled studies examine accuracy outcomes, feedback quality, and learner engagement together. This study addresses that gap through a controlled investigation conducted in an under-represented South Asian English-as-a-foreign-language (EFL) setting. Adopting a sequential explanatory mixed-methods design, ninety undergraduate EFL learners were stratified and randomly assigned to three conditions—GPT-4 AWCF, expert teacher-written corrective feedback, and a self-revision control—across an eight-week, six-cycle argumentative writing programme. Quantitative strands comprised error-rate analysis, an analytic rubric, and a validated engagement questionnaire; qualitative strands comprised corpus coding of 1,860 feedback moves, stimulated-recall interviews, and reflective journals, integrated through a joint display. Analysis of covariance indicated that the GPT-4 group significantly outperformed the control on both accuracy and rubric scores, with gains statistically comparable to, and on local accuracy marginally exceeding, the teacher group, and with most gains retained at delayed post-testing. Feedback coding revealed that GPT-4 favoured metalinguistic explanation and reformulation but showed reduced precision on discourse-level concerns, where redundancy and occasional hallucinated corrections appeared. Engagement was high—particularly on the affective dimension—but cognitively demanding, and learners expressed concern about over-reliance and loss of authorial voice. The study contributes controlled, integrated evidence on the pedagogical value and limitations of LLM-based language technology and argues for a principled human-in-the-loop model of AI-assisted feedback rather than wholesale automation.

Overview

CoChat Summary

TL;DR

This mixed-methods study found that GPT-4 automated written corrective feedback improved EFL academic writing accuracy and quality comparably to expert teacher feedback and significantly better than self-revision, with most gains retained at delayed post-testing. Feedback quality was strong on local correction but weaker on discourse-level concerns, and learners reported high affective engagement alongside cognitive load and over-reliance worries.

Key findings

  • ANCOVA showed significant condition effects on post-test accuracy, F(2, 86) = 21.4, p < .001, partial η² = .33, and rubric scores, F(2, 86) = 17.8, p < .001, partial η² = .29. Both feedback groups outperformed the control (p < .001); GPT-4 and teacher groups did not differ significantly on rubric scores, while GPT-4 showed marginally larger local accuracy gains (errors per 100 words: GPT-4 9.84 to 5.12; teacher 9.91 to 5.64; control 9.78 to 8.41).
  • Coding of 1,860 feedback moves showed GPT-4 gave more metalinguistic explanation (34.7%) and reformulation (23.5%), while teachers gave more direct correction (38.2%) and content/organisation comments (13.7%). GPT-4 accuracy was lower on content/organisation (71.2%) than teacher accuracy (93.4%), with some redundant or hallucinated corrections; local grammar/lexis accuracy was high for both sources (Cohen’s κ = .81).
  • Engagement was high in both feedback groups. GPT-4 had significantly higher affective engagement (4.46 vs. 3.88; t = 3.85, p < .001), with no significant differences in behavioural (4.31 vs. 4.18, p = .36) or cognitive (4.05 vs. 3.97, p = .62) engagement; overall was 4.27 vs. 4.01 (p = .05). Qualitative themes included on-demand immediacy (27 sources), over-reliance and ownership concerns (19), discourse-level unevenness (22), and preference for blended human–AI feedback (24).

Methods

  • Sequential explanatory mixed-methods design with 90 undergraduate EFL learners in Lahore, Pakistan, stratified and randomly assigned to GPT-4 AWCF, expert teacher WCF, or self-revision control (n = 30 each), across an eight-week, six-cycle argumentative writing programme.
  • Quantitative strands used error-rate analysis, an analytic rubric (0–25), and a validated engagement questionnaire; analyses included ANCOVA with pre-test covariate, repeated-measures ANOVA for delayed post-test retention, and independent-samples t-tests.
  • Qualitative strands coded 1,860 feedback moves, stimulated-recall and semi-structured interviews, and reflective journals, integrated through a joint display; GPT-4 used a standardised version-controlled prompt template, teachers used a standardised annotation protocol, and controls used a self-editing checklist.

Limitations

  • The paper states the study was conducted in a single context, over an eight-week period, with a single model, a controlled prompt configuration, and partially self-reported engagement; a single cohort limits generalisation.
  • The paper notes that controlled elicitation sacrifices some ecological validity for internal validity, since learners did not formulate their own prompts.
  • The paper calls for future work using other models and disciplines, longer time horizons, and process data such as keystroke and revision logs; the text makes evident that engagement findings rely partly on self-report rather than behavioural process measures.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.57041/x2hg0114

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Sequential explanatory mixed-methods research design.
Fig. 1 Sequential explanatory mixed-methods research design.From the PDF

Verification

Claims the paper makes

4 Valid · 0 Not valid · 28 Not checked · 65 Own results and interpretations

24 more claims

Create a free account to see all 97 claims and to verify the 8 references still open.

See all claims — free account
  • Probably not a substantial claim. 31
  • It may state two separate claims as one. 54
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(2, 86) = 21.4reported p < .001 · recomputed p < .001

    “F(2, 86) = 21.4, p < .001”

  • ConsistentF(2, 86) = 17.8reported p < .001 · recomputed p < .001

    “F(2, 86) = 17.8, p < .001”

2 of 2

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 20 references

Metadata

Authors
Pasha, M. K.
Affiliations
University of Engineering and Technology Lahore (PK)
Corresponding author
Mujtaba Kamal Pasha
Venue
Journal of Artificial Intelligence and Computing
Published
2026-06-29
Volume
4
Issue
1
Pages
8-18
ISSN
3005-3358 (ISSN-L)
OpenAlex
W7168124920
Cited by
0
References
22
FWCI
0.00
Type
journal article
Language
en
Open access
Locations
Journal of Artificial Intelligence and Computing (publishedVersion, cc-by-sa)
Topics
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
  • Social Sciences › Education › English Language Learning and Teaching
Keywords
generative AI, EFL academic writing, GPT-4, large language models, teacher feedback, student engagement, metalinguistic explanation, hallucinations, human-in-the-loop, AI-assisted writing, argumentative writing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Generative AI as an Automated Written Corrective Feedback Provider in EFL Academic Writing: A Mixed-Methods Investigation of Accuracy Gains, Feedback Quality, and Learner Engagement. https://app.cochat.ai/papers/10.57041/x2hg0114
The paper
Pasha, M. K. (2026). Generative AI as an Automated Written Corrective Feedback Provider in EFL Academic Writing: A Mixed-Methods Investigation of Accuracy Gains, Feedback Quality, and Learner Engagement. Journal of Artificial Intelligence and Computing. https://doi.org/10.57041/x2hg0114

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.