Generative AI as an Automated Written Corrective Feedback Provider in EFL Academic Writing: A Mixed-Methods Investigation of Accuracy Gains, Feedback Quality, and Learner Engagement
Open accessThe DOI matches its Crossref record; 20 of the 22 references it cites were found in a registry or an index; 4 of the 46 sources its claims cite were checked for support.
Abstract
The integration of generative artificial intelligence (GenAI) and large language models (LLMs) into language technology has reopened long-standing questions about the role of automated written corrective feedback (AWCF) in second-language (L2) writing development. While early enthusiasm has positioned conversational LLMs such as GPT-4 as scalable substitutes for the labour-intensive provision of teacher feedback, the empirical base remains fragmented, and few controlled studies examine accuracy outcomes, feedback quality, and learner engagement together. This study addresses that gap through a controlled investigation conducted in an under-represented South Asian English-as-a-foreign-language (EFL) setting. Adopting a sequential explanatory mixed-methods design, ninety undergraduate EFL learners were stratified and randomly assigned to three conditions—GPT-4 AWCF, expert teacher-written corrective feedback, and a self-revision control—across an eight-week, six-cycle argumentative writing programme. Quantitative strands comprised error-rate analysis, an analytic rubric, and a validated engagement questionnaire; qualitative strands comprised corpus coding of 1,860 feedback moves, stimulated-recall interviews, and reflective journals, integrated through a joint display. Analysis of covariance indicated that the GPT-4 group significantly outperformed the control on both accuracy and rubric scores, with gains statistically comparable to, and on local accuracy marginally exceeding, the teacher group, and with most gains retained at delayed post-testing. Feedback coding revealed that GPT-4 favoured metalinguistic explanation and reformulation but showed reduced precision on discourse-level concerns, where redundancy and occasional hallucinated corrections appeared. Engagement was high—particularly on the affective dimension—but cognitively demanding, and learners expressed concern about over-reliance and loss of authorial voice. The study contributes controlled, integrated evidence on the pedagogical value and limitations of LLM-based language technology and argues for a principled human-in-the-loop model of AI-assisted feedback rather than wholesale automation.
Overview
CoChat Summary
TL;DR
This mixed-methods study found that GPT-4 automated written corrective feedback improved EFL academic writing accuracy and quality comparably to expert teacher feedback and significantly better than self-revision, with most gains retained at delayed post-testing. Feedback quality was strong on local correction but weaker on discourse-level concerns, and learners reported high affective engagement alongside cognitive load and over-reliance worries.
Key findings
- ANCOVA showed significant condition effects on post-test accuracy, F(2, 86) = 21.4, p < .001, partial η² = .33, and rubric scores, F(2, 86) = 17.8, p < .001, partial η² = .29. Both feedback groups outperformed the control (p < .001); GPT-4 and teacher groups did not differ significantly on rubric scores, while GPT-4 showed marginally larger local accuracy gains (errors per 100 words: GPT-4 9.84 to 5.12; teacher 9.91 to 5.64; control 9.78 to 8.41).
- Coding of 1,860 feedback moves showed GPT-4 gave more metalinguistic explanation (34.7%) and reformulation (23.5%), while teachers gave more direct correction (38.2%) and content/organisation comments (13.7%). GPT-4 accuracy was lower on content/organisation (71.2%) than teacher accuracy (93.4%), with some redundant or hallucinated corrections; local grammar/lexis accuracy was high for both sources (Cohen’s κ = .81).
- Engagement was high in both feedback groups. GPT-4 had significantly higher affective engagement (4.46 vs. 3.88; t = 3.85, p < .001), with no significant differences in behavioural (4.31 vs. 4.18, p = .36) or cognitive (4.05 vs. 3.97, p = .62) engagement; overall was 4.27 vs. 4.01 (p = .05). Qualitative themes included on-demand immediacy (27 sources), over-reliance and ownership concerns (19), discourse-level unevenness (22), and preference for blended human–AI feedback (24).
Methods
- Sequential explanatory mixed-methods design with 90 undergraduate EFL learners in Lahore, Pakistan, stratified and randomly assigned to GPT-4 AWCF, expert teacher WCF, or self-revision control (n = 30 each), across an eight-week, six-cycle argumentative writing programme.
- Quantitative strands used error-rate analysis, an analytic rubric (0–25), and a validated engagement questionnaire; analyses included ANCOVA with pre-test covariate, repeated-measures ANOVA for delayed post-test retention, and independent-samples t-tests.
- Qualitative strands coded 1,860 feedback moves, stimulated-recall and semi-structured interviews, and reflective journals, integrated through a joint display; GPT-4 used a standardised version-controlled prompt template, teachers used a standardised annotation protocol, and controls used a self-editing checklist.
Limitations
- The paper states the study was conducted in a single context, over an eight-week period, with a single model, a controlled prompt configuration, and partially self-reported engagement; a single cohort limits generalisation.
- The paper notes that controlled elicitation sacrifices some ecological validity for internal validity, since learners did not formulate their own prompts.
- The paper calls for future work using other models and disciplines, longer time horizons, and process data such as keystroke and revision logs; the text makes evident that engagement findings rely partly on self-report rather than behavioural process measures.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 0 Not valid · 28 Not checked · 65 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Feedback coding revealed that GPT-4 favoured metalinguistic explanation and reformulation”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, engagement was high in both feedback groups”
plainn = 30- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and with most gains retained at delayed post-testing”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Analysis of covariance indicated that the GPT-4 group significantly outperformed the control on both accuracy and rubric scores”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Analysis of covariance indicated that the GPT-4 group significantly outperformed the control on both accuracy and rubric scores”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
24 more claims
Create a free account to see all 97 claims and to verify the 8 references still open.
- Probably not a substantial claim. 31
- It may state two separate claims as one. 54
- The quote only partly states this claim. 1
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(2, 86) = 21.4reported p < .001 · recomputed p < .001
“F(2, 86) = 21.4, p < .001”
- ConsistentF(2, 86) = 17.8reported p < .001 · recomputed p < .001
“F(2, 86) = 17.8, p < .001”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ B. Empirical effects of LLM feedback on writing · p. 3
“Liu in [14] found that, in the context of EFL writing education, integrating an LLM-based model into the learning process led to substantial improvements in writing, self-regulated learning strategy use, and writing motivation in a randomized study”
§ REFERENCES · p. 10
“[14] Liu, Y., Lu, X., & Qi, H., Comparing GPT-based approaches in automated writing evaluation.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Generative AI as an Automated Written Corrective Feedback Provider in EFL Academic Writing: A Mixed-Methods Investigation of Accuracy Gains, Feedback Quality, and Learner Engagement · p. 1
“Mujtaba Kamal Pasha”
§ AUTHOR CONTRIBUTIONS · p. 10
“Conceptualization, methodology, validation, writing -original draft preparation, writing -review and editing, E.K., I.A.”
Citations
Cites
Guo, K., & Wang, D. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12146-0 · Cited by 449 · cited 3×
“The second is the displacement of teacher expertise, and thus the suggestion in literature is for complementary configurations where AI is used to complement rather than replace teacher feedback [17].”
Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study
Mahapatra, S. (2024) · Smart Learning Environments · doi:10.1186/s40561-024-00295-9 · Cited by 398 · cited 3×
“In a mixed-methods intervention with tertiary ESL learners, Mahapatra [6] reported that ChatGPT as a formative feedback tool significantly improves academic writing skills and that it was overwhelmingly well-received by the students.”
Teng, M. F. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100270 · Cited by 182 · cited 4×
“The presence of affective advantage reflects previous studies on engagement and perception [2], [7], and the higher cognitive load aligns with the fact that the effectiveness of feedback is actually mediated by the quality of engagement [3].”
InfluentialPack, A., Barrett, A., & Escalante, J. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100234 · Cited by 166 · cited 4×
“This asymmetry aligns with previous comparative results [4] and the general trend in the automated-scoring literature, that the validity of LLMs is determined by model, rubric, task, and population and not by surface fluency [9], [16], [5].”
InfluentialGenerative AI (GenAI) in the language classroom: A systematic review
Lee, S., Choe, H., Zou, D., & Jeon, J. (2026) · Interactive Learning Environments · doi:10.1080/10494820.2025.2498537 · Cited by 158 · cited 3×
“It also warns against interpreting any single estimate of the effect, such as the one presented here, as an absolute property of the technology and not one generated during the situation [15], [12].”
Li, J., Huang, J., Wu, W., & Whipple, P. B. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03755-2 · Cited by 131 · cited 2×
“It also highlights the need to validate rather than simply take for granted assessment-oriented applications of GenAI and the defensibility of framing such applications as a way to reinvent, not just accelerate, assessment [18], [19], [10].”
Liu, Z.-M., Hwang, G.-J., Chen, C.-Q., Chen, X.-D., & Ye, X.-D. (2026) · Computer Assisted Language Learning · doi:10.1080/09588221.2024.2389923 · Cited by 115 · cited 4×
“In the realm of early intervention, positive outcomes on writing performance [5], [6], self-regulated learning and motivation [6] have been reported when LLMs are systematically integrated into instruction, while learners' perceptions of LLMs as supportive, non-threatening feedback companions have…”
InfluentialDu, J., & Alm, A. (2024) · Education Sciences · doi:10.3390/educsci14070726 · Cited by 115 · cited 3×
“The most evident behavioural footprint was the affective benefit of AI feedback — its evaluating-free nature and its immediacy — which confirmed previous case study and perception research [2], [7], [8] and the self-determination account of LLM appeal.”
Bui, N. M., & Barrot, J. S. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-12891-w · Cited by 113 · cited 2×
“This profile validates and builds upon previous comparative results [4] and the cautions to validity found in the essay-scoring literature [9], [18].”
Yan, D., & Zhang, S. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03543-y · Cited by 90
The grass is not always greener: Teacher vs. GPT-assisted written corrective feedback
Lin, S., & Crosthwaite, P. (2024) · System · doi:10.1016/j.system.2024.103529 · Cited by 86 · cited 5×
“Unlike previous AWE tools that generate fixed templates, LLMs can offer contextual, explanatory, and dialogic responses, which can serve as a metalinguistic commentary from a human tutor and can be questioned, edited, and re-prompted by the learner [4].”
InfluentialFoung, D., Lin, L., & Chen, J. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100250 · Cited by 54 · cited 1×
“This setting is the most consistently suggested in the literature of the review and comparison [17], [20], [15], [13], [12].”
Generative AI and L2 Written Feedback Studies: A Scoping Review
Crosthwaite, P., & Sun, S. (2026) · RELC Journal · doi:10.1177/00336882251386530 · Cited by 50 · cited 4×
“It also warns against interpreting any single estimate of the effect, such as the one presented here, as an absolute property of the technology and not one generated during the situation [15], [12].”
InfluentialApplying large language models for automated essay scoring for non-native Japanese
Li, W., & Liu, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03209-9 · Cited by 36 · cited 2×
“This asymmetry aligns with previous comparative results [4] and the general trend in the automated-scoring literature, that the validity of LLMs is determined by model, rubric, task, and population and not by surface fluency [9], [16], [5].”
Guo, X. (2025) · Interactive Learning Environments · doi:10.1080/10494820.2024.2445177 · Cited by 23 · cited 4×
“The second, over-reliance and ownership, expressed a concern in the opposite direction: some students feared that they were blindly copying ideas and that the authorial voice of the model was eroding their own – a fear which explicitly mirrors the thinking-inhibitor critique [11].”
InfluentialA systematic review of Grammarly in L2 English writing contexts
Dizon, G., & Gayed, J. M. (2024) · Cogent Education · doi:10.1080/2331186x.2024.2397882 · Cited by 19
Comparing GPT-based approaches in automated writing evaluation
Liu, Y., Lu, X., & Qi, H. (2025) · Assessing Writing · doi:10.1016/j.asw.2025.100961 · Cited by 19 · cited 1×
“Liu in [14] found that, in the context of EFL writing education, integrating an LLM-based model into the learning process led to substantial improvements in writing, self-regulated learning strategy use, and writing motivation in a randomized study, highlighting the crucial role of human teachers…”
On the role of engagement in automated feedback effectiveness: Insights from keystroke logging
Schiller, R., Fleckenstein, J., Höft, L., Horbach, A., & Meyer, J. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105386 · Cited by 14 · cited 3×
“Process-oriented research validated the mediating role of engagement: Schiller [3] demonstrated that the effectiveness of automated feedback is not simply because it is available, but actually because of a material engagement with it, using keystroke logging.”
Exploring the Use of ChatGPT as a Tool for Written Corrective Feedback in an EFL Classroom
Education Bureau of Hong Kong, S. A. R., & Long, H. S. (2024) · The Journal of AsiaTEFL · doi:10.18823/asiatefl.2024.21.2.8.397 · Cited by 9 · cited 2×
“ChatGPT has demonstrated clear potential for use as a classroom WCF tool, but initial classroom studies have also found that feedback varies in quantity and quality over time [13].”
Pack, A., Hartshorn, K. J., Escalante, J., & Gillette, N. (2025) · International Journal of English for Academic Purposes: Research and Practice · doi:10.3828/ijeap.2025.2 · Cited by 3 · cited 2×
“…output will remain unknown: Research on automated essay scoring and AI-generated feedback shows that validity and reliability will be highly dependent on the model, rubric, task, and population and that the apparent fluency of LLM responses can mask mistakes and even invented corrections [9], [10].”
Metadata
- Authors
- Pasha, M. K.
- Affiliations
- University of Engineering and Technology Lahore (PK)
- Corresponding author
- Mujtaba Kamal Pasha
- Venue
- Journal of Artificial Intelligence and Computing
- Published
- 2026-06-29
- Volume
- 4
- Issue
- 1
- Pages
- 8-18
- ISSN
- 3005-3358 (ISSN-L)
- OpenAlex
- W7168124920
- Semantic Scholar
- 6241708424cefa5c35ca9dff236ba25956189f86
- Cited by
- 0
- References
- 22
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Artificial Intelligence and Computing (publishedVersion, cc-by-sa)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
- Social Sciences › Education › English Language Learning and Teaching
- Keywords
- generative AI, EFL academic writing, GPT-4, large language models, teacher feedback, student engagement, metalinguistic explanation, hallucinations, human-in-the-loop, AI-assisted writing, argumentative writing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
