Generative AI as an Automated Written Corrective Feedback Provider in EFL Academic Writing: A Mixed-Methods Investigation of Accuracy Gains, Feedback Quality, and Learner Engagement
Open accessThe DOI matches its Crossref record; 20 of the 22 references it cites were found in a registry or an index; 4 of the 46 sources its claims cite were checked for support.
Abstract
The integration of generative artificial intelligence (GenAI) and large language models (LLMs) into language technology has reopened long-standing questions about the role of automated written corrective feedback (AWCF) in second-language (L2) writing development. While early enthusiasm has positioned conversational LLMs such as GPT-4 as scalable substitutes for the labour-intensive provision of teacher feedback, the empirical base remains fragmented, and few controlled studies examine accuracy outcomes, feedback quality, and learner engagement together. This study addresses that gap through a controlled investigation conducted in an under-represented South Asian English-as-a-foreign-language (EFL) setting. Adopting a sequential explanatory mixed-methods design, ninety undergraduate EFL learners were stratified and randomly assigned to three conditions—GPT-4 AWCF, expert teacher-written corrective feedback, and a self-revision control—across an eight-week, six-cycle argumentative writing programme. Quantitative strands comprised error-rate analysis, an analytic rubric, and a validated engagement questionnaire; qualitative strands comprised corpus coding of 1,860 feedback moves, stimulated-recall interviews, and reflective journals, integrated through a joint display. Analysis of covariance indicated that the GPT-4 group significantly outperformed the control on both accuracy and rubric scores, with gains statistically comparable to, and on local accuracy marginally exceeding, the teacher group, and with most gains retained at delayed post-testing. Feedback coding revealed that GPT-4 favoured metalinguistic explanation and reformulation but showed reduced precision on discourse-level concerns, where redundancy and occasional hallucinated corrections appeared. Engagement was high—particularly on the affective dimension—but cognitively demanding, and learners expressed concern about over-reliance and loss of authorial voice. The study contributes controlled, integrated evidence on the pedagogical value and limitations of LLM-based language technology and argues for a principled human-in-the-loop model of AI-assisted feedback rather than wholesale automation.
Overview
CoChat Summary
TL;DR
This mixed-methods study found that GPT-4 automated written corrective feedback improved EFL academic writing accuracy and quality comparably to expert teacher feedback and significantly better than self-revision, with most gains retained at delayed post-testing. Feedback quality was strong on local correction but weaker on discourse-level concerns, and learners reported high affective engagement alongside cognitive load and over-reliance worries.
Key findings
- ANCOVA showed significant condition effects on post-test accuracy, F(2, 86) = 21.4, p < .001, partial η² = .33, and rubric scores, F(2, 86) = 17.8, p < .001, partial η² = .29. Both feedback groups outperformed the control (p < .001); GPT-4 and teacher groups did not differ significantly on rubric scores, while GPT-4 showed marginally larger local accuracy gains (errors per 100 words: GPT-4 9.84 to 5.12; teacher 9.91 to 5.64; control 9.78 to 8.41).
- Coding of 1,860 feedback moves showed GPT-4 gave more metalinguistic explanation (34.7%) and reformulation (23.5%), while teachers gave more direct correction (38.2%) and content/organisation comments (13.7%). GPT-4 accuracy was lower on content/organisation (71.2%) than teacher accuracy (93.4%), with some redundant or hallucinated corrections; local grammar/lexis accuracy was high for both sources (Cohen’s κ = .81).
- Engagement was high in both feedback groups. GPT-4 had significantly higher affective engagement (4.46 vs. 3.88; t = 3.85, p < .001), with no significant differences in behavioural (4.31 vs. 4.18, p = .36) or cognitive (4.05 vs. 3.97, p = .62) engagement; overall was 4.27 vs. 4.01 (p = .05). Qualitative themes included on-demand immediacy (27 sources), over-reliance and ownership concerns (19), discourse-level unevenness (22), and preference for blended human–AI feedback (24).
Methods
- Sequential explanatory mixed-methods design with 90 undergraduate EFL learners in Lahore, Pakistan, stratified and randomly assigned to GPT-4 AWCF, expert teacher WCF, or self-revision control (n = 30 each), across an eight-week, six-cycle argumentative writing programme.
- Quantitative strands used error-rate analysis, an analytic rubric (0–25), and a validated engagement questionnaire; analyses included ANCOVA with pre-test covariate, repeated-measures ANOVA for delayed post-test retention, and independent-samples t-tests.
- Qualitative strands coded 1,860 feedback moves, stimulated-recall and semi-structured interviews, and reflective journals, integrated through a joint display; GPT-4 used a standardised version-controlled prompt template, teachers used a standardised annotation protocol, and controls used a self-editing checklist.
Limitations
- The paper states the study was conducted in a single context, over an eight-week period, with a single model, a controlled prompt configuration, and partially self-reported engagement; a single cohort limits generalisation.
- The paper notes that controlled elicitation sacrifices some ecological validity for internal validity, since learners did not formulate their own prompts.
- The paper calls for future work using other models and disciplines, longer time horizons, and process data such as keystroke and revision logs; the text makes evident that engagement findings rely partly on self-report rather than behavioural process measures.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 0 Not valid · 28 Not checked · 65 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Feedback coding revealed that GPT-4 favoured metalinguistic explanation and reformulation”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, engagement was high in both feedback groups”
plainn = 30- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and with most gains retained at delayed post-testing”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Analysis of covariance indicated that the GPT-4 group significantly outperformed the control on both accuracy and rubric scores”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Analysis of covariance indicated that the GPT-4 group significantly outperformed the control on both accuracy and rubric scores”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
24 more claims
Create a free account to see all 97 claims and to verify the 8 references still open.
- Probably not a substantial claim. 31
- It may state two separate claims as one. 54
- The quote only partly states this claim. 1
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(2, 86) = 21.4reported p < .001 · recomputed p < .001
“F(2, 86) = 21.4, p < .001”
- ConsistentF(2, 86) = 17.8reported p < .001 · recomputed p < .001
“F(2, 86) = 17.8, p < .001”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ B. Empirical effects of LLM feedback on writing · p. 3
“Liu in [14] found that, in the context of EFL writing education, integrating an LLM-based model into the learning process led to substantial improvements in writing, self-regulated learning strategy use, and writing motivation in a randomized study”
§ REFERENCES · p. 10
“[14] Liu, Y., Lu, X., & Qi, H., Comparing GPT-based approaches in automated writing evaluation.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Generative AI as an Automated Written Corrective Feedback Provider in EFL Academic Writing: A Mixed-Methods Investigation of Accuracy Gains, Feedback Quality, and Learner Engagement · p. 1
“Mujtaba Kamal Pasha”
§ AUTHOR CONTRIBUTIONS · p. 10
“Conceptualization, methodology, validation, writing -original draft preparation, writing -review and editing, E.K., I.A.”
Citations
Cites
Guo, K., & Wang, D. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12146-0 · Cited by 449 · cited 3×
“The second is the displacement of teacher expertise, and thus the suggestion in literature is for complementary configurations where AI is used to complement rather than replace teacher feedback [17].”
Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study
Mahapatra, S. (2024) · Smart Learning Environments · doi:10.1186/s40561-024-00295-9 · Cited by 398 · cited 3×
“In a mixed-methods intervention with tertiary ESL learners, Mahapatra [6] reported that ChatGPT as a formative feedback tool significantly improves academic writing skills and that it was overwhelmingly well-received by the students.”
Teng, M. F. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100270 · Cited by 182 · cited 4×
“The presence of affective advantage reflects previous studies on engagement and perception [2], [7], and the higher cognitive load aligns with the fact that the effectiveness of feedback is actually mediated by the quality of engagement [3].”
InfluentialPack, A., Barrett, A., & Escalante, J. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100234 · Cited by 166 · cited 4×
“This asymmetry aligns with previous comparative results [4] and the general trend in the automated-scoring literature, that the validity of LLMs is determined by model, rubric, task, and population and not by surface fluency [9], [16], [5].”
InfluentialGenerative AI (GenAI) in the language classroom: A systematic review
Lee, S., Choe, H., Zou, D., & Jeon, J. (2026) · Interactive Learning Environments · doi:10.1080/10494820.2025.2498537 · Cited by 158 · cited 3×
“It also warns against interpreting any single estimate of the effect, such as the one presented here, as an absolute property of the technology and not one generated during the situation [15], [12].”
Li, J., Huang, J., Wu, W., & Whipple, P. B. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03755-2 · Cited by 131 · cited 2×
“It also highlights the need to validate rather than simply take for granted assessment-oriented applications of GenAI and the defensibility of framing such applications as a way to reinvent, not just accelerate, assessment [18], [19], [10].”
Liu, Z.-M., Hwang, G.-J., Chen, C.-Q., Chen, X.-D., & Ye, X.-D. (2026) · Computer Assisted Language Learning · doi:10.1080/09588221.2024.2389923 · Cited by 115 · cited 4×
“In the realm of early intervention, positive outcomes on writing performance [5], [6], self-regulated learning and motivation [6] have been reported when LLMs are systematically integrated into instruction, while learners' perceptions of LLMs as supportive, non-threatening feedback companions have…”
InfluentialDu, J., & Alm, A. (2024) · Education Sciences · doi:10.3390/educsci14070726 · Cited by 115 · cited 3×
“The most evident behavioural footprint was the affective benefit of AI feedback — its evaluating-free nature and its immediacy — which confirmed previous case study and perception research [2], [7], [8] and the self-determination account of LLM appeal.”
Bui, N. M., & Barrot, J. S. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-12891-w · Cited by 113 · cited 2×
“This profile validates and builds upon previous comparative results [4] and the cautions to validity found in the essay-scoring literature [9], [18].”
Yan, D., & Zhang, S. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03543-y · Cited by 90
Metadata
- Authors
- Pasha, M. K.
- Affiliations
- University of Engineering and Technology Lahore (PK)
- Corresponding author
- Mujtaba Kamal Pasha
- Venue
- Journal of Artificial Intelligence and Computing
- Published
- 2026-06-29
- Volume
- 4
- Issue
- 1
- Pages
- 8-18
- ISSN
- 3005-3358 (ISSN-L)
- OpenAlex
- W7168124920
- Semantic Scholar
- 6241708424cefa5c35ca9dff236ba25956189f86
- Cited by
- 0
- References
- 22
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Artificial Intelligence and Computing (publishedVersion, cc-by-sa)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
- Social Sciences › Education › English Language Learning and Teaching
- Keywords
- generative AI, EFL academic writing, GPT-4, large language models, teacher feedback, student engagement, metalinguistic explanation, hallucinations, human-in-the-loop, AI-assisted writing, argumentative writing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generative AI as an Automated Written Corrective Feedback Provider in EFL Academic Writing: A Mixed-Methods Investigation of Accuracy Gains, Feedback Quality, and Learner Engagement. https://app.cochat.ai/papers/10.57041/x2hg0114
- The paper
- Pasha, M. K. (2026). Generative AI as an Automated Written Corrective Feedback Provider in EFL Academic Writing: A Mixed-Methods Investigation of Accuracy Gains, Feedback Quality, and Learner Engagement. Journal of Artificial Intelligence and Computing. https://doi.org/10.57041/x2hg0114
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
