Learner interaction with, and response to, AI-programmed automated writing evaluation feedback in EFL writing: An exploratory study
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Recently, artificial intelligence (AI)-programmed automated writing evaluation (AWE) has attracted increasing attention in language research. Using a small data set arising from an analysis of five Chinese university-level English as a foreign language (EFL) students’ submissions, this paper examined in detail how EFL students interacted with the feedback of Pigai, the largest AI-programmed AWE in China. The analysis started with the intention of capturing the machine feedback on the five students’ submissions and the exchanges between the participants and Pigai over repeated submissions, ranging from 3 to 12 submissions. The analysis showed that the learners’ interactions with Pigai focused on error corrective feedback in the initial two submissions. In the case of one student who had 12 submissions, the non-error corrective feedback increased gradually over time, providing rich linguistic resources but without examples and contextual information. The students’ take-up rates of feedback with linguistic resources were much lower than that of error corrective and general feedback. A terrain model to map the stages and nature of student responses showed a more complete dynamic process, in which students’ responses changed from the initial mechanical responses at the discrete language level to more considered approaches in response to machine feedback. The findings of this study have implications for both language pedagogy and the future design and development of AWE for second or foreign language learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ferris, D. (2006) · Feedback in Second Language Writing · doi:10.1017/cbo9781139524742.007 · Cited by 662
Student engagement with teacher and automated feedback on L2 writing
Zhang, Z. (., & Hyland, K. (2018) · Assessing Writing · doi:10.1016/j.asw.2018.02.004 · Cited by 546
Koltovskaia, S. (2020) · Assessing Writing · doi:10.1016/j.asw.2020.100450 · Cited by 447 · cited 5×
“One strategy is to combine the teacher, peer and AWE feedback in the revision process (Koltovskaia, 2020; Zhang & Hyland, 2022).”
InfluentialFocusing on form: student engagement with teacher feedback?
Hyland, F. (2003) · System · doi:10.1016/s0346-251x(03)00021-6 · Cited by 411 · cited 3×
“To address the second research question, students’ revisions were analysed in three rounds according to the coding schemes from Ferris (2006) and Hyland (2003) and the refined coding in Han and Hyland (2015), regarding students’ revision based on teacher WCF (Appendix Table 10).”
The effects of computer-generated feedback on the quality of writing
Stevenson, M., & Phakiti, A. (2014) · Assessing Writing · doi:10.1016/j.asw.2013.11.007 · Cited by 275 · cited 3×
“Research on AWE has largely focused on writing products, with little attention paid to the revision process (Stevenson & Phakiti, 2014; Storch, 2018).”
Wang, Y.-J., Shang, H.-F., & Briody, P. (2013) · Computer Assisted Language Learning · doi:10.1080/09588221.2012.655300 · Cited by 185 · cited 4×
“…could lead to improvement in students’ writing especially in terms of error rate reduction (Al-Inbari & Al-Wasy, 2022; Liao, 2016; Lu, 2019; Wang et al, 2013), with reduced frequency of grammatical, vocabulary, spelling, and conjunction errors was observed (Liao, 2016; Song, 2019; Tan,…”
InfluentialThe impact of computer-based feedback on students' written work?
El Ebyary, K., & Windeatt, S. (2010) · International Journal of English Studies · doi:10.6018/ijes/2010/2/119231 · Cited by 94 · cited 2×
“…with AWE have been identified as (1) superficial correction/revision with low uptake (Bai & Hu, 2017); (2) very limited time (average 6 min) spent on revision after receiving AWE feedback (Warden, 2000); and (3) no revision after receiving feedback (El Ebyary & Windeatt, 2010; Jiang, 2015).”
EFL business writing behaviors in differing feedback environments
Warden, C. A. (2000) · Language Learning · doi:10.1111/0023-8333.00141 · Cited by 34 · cited 1×
“…with AWE have been identified as (1) superficial correction/revision with low uptake (Bai & Hu, 2017); (2) very limited time (average 6 min) spent on revision after receiving AWE feedback (Warden, 2000); and (3) no revision after receiving feedback (El Ebyary & Windeatt, 2010; Jiang, 2015).”
Cited by
Karataş, F., Abedi, F. Y., Ozek Gunyel, F., Karadeniz, D., & Kuzgun, Y. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12574-6 · Cited by 252
Ba, S., Yang, L., Yan, Z., Looi, C. K., & Gašević, D. (2025) · Computers and Education Open · doi:10.1016/j.caeo.2025.100284 · Cited by 91
Tran, T. T. T. (2025) · Education Sciences · doi:10.3390/educsci15020232 · Cited by 68
“For instance, Yang et al. (2024) examined in detail how five EFL students interacted with technology-based feedback, highlighting the challenges of maintaining participant engagement throughout the study.”
Unpacking the Rejection of L2 Students Toward ChatGPT-Generated Feedback: An Explanatory Research
Chen (陈子淇), Z., Wei (韦卫), W., Zhu (祝新华), X., & Yao (姚远), Y. (2026) · ECNU Review of Education · doi:10.1177/20965311241305140 · Cited by 16
“…received feedback (Cheong et al., 2023): (1) accept and revise accordingly and (2) show a certain level of rejection, such as ignorance, disagreement, and confusion by arguing with the feedback providers, especially for non-error corrective feedback (Mohebbi, 2021; Yang et al., 2023; Zhang, 2020).”
Bo, Z., Pek, L. S., Jian, L., Cong, W., & Nan, L. T. (2025) · Language Teaching Research Quarterly · doi:10.32038/ltrq.2025.53.05 · Cited by 6
Metadata
- Authors
- Yang, H., Gao, C., & Shen, H.-Z.
- Venue
- Education and Information Technologies
- Published
- 2023-06-27
- Volume
- 29
- Issue
- 4
- Pages
- 3837-3858
- ISSN
- 1360-2357 (ISSN-L)
- 1573-7608
- OpenAlex
- W4382317804
- Cited by
- 86
- References
- 8
- FWCI
- 98.89
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Learner interaction with, and response to, AI-programmed automated writing evaluation feedback in EFL writing: An exploratory study. https://app.cochat.ai/papers/10.1007/s10639-023-11991-3
- The paper
- Yang, H., Gao, C., & Shen, H.-Z. (2024). Learner interaction with, and response to, AI-programmed automated writing evaluation feedback in EFL writing: An exploratory study. Education and Information Technologies. https://doi.org/10.1007/s10639-023-11991-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.