From process to product: writing engagement and performance of EFL learners under computer-generated feedback instruction
Open accessThe DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
Artificial intelligence (AI) technology is gradually penetrating the domain of education, opening up many possibilities for teaching and learning. Many educators, faced with the burden of commenting on substantial student essays, have introduced automated writing evaluation (AWE) into second language (L2) writing considering its affordance of immediate scores and diagnostic information. However, students’ processing strategies and perceptions of such computer-generated feedback and its impact on student writing quality, particularly as mediated by language proficiency, remain under-explored. This study examines the impact of Pigai , a Chinese AWE system, on revision processes and writing products of 42 English as a foreign language (EFL) learners with varying language levels by analyzing feedback points, feedback uptake, text quality in complexity, accuracy, and fluency (CAF), and perceptions. The findings are as follows. When confronted with AWE instruction, the majority of student work focuses on correcting errors, but higher-level students exhibit an emphasis on language improvement beyond the surface level compared to lower-level students. According to CAF measures, automated feedback exerts greater effects on accuracy for unskilled learners and lexical complexity for skilled learners in the development of interlanguage. Learner profiles and perceptions of students at different levels are explored along four dimensions: writing quality, cognitive engagement, behavioral engagement, and affective engagement. Finally, the potential issues of such technology-based writing instruction are pointed out.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, AI tools provide immediate, individualized diagnostic feedback on linguistic accuracy, syntactic complexity, and genre-level organization—otherwise constrained by class size and instructor availability—enabling students to identify and repair surface and structural weaknesses across multiple drafting cycles (Chan et al., 2024; Mekheimer, 2025; Shen et al., 2023; Song and Song, 2023).”
From From tool to scaffold: structured human–AI collaboration and its effects on academic writing and digital critical thinking among Saudi EFL learners (2026) · cites this paper as Shen et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Few studies have looked at the long-term effects of ChatGPT and similar tools on the development of writing skills, particularly in actual classroom settings.”
From Evaluating the impact of using ChatGPT on EFL students’ argumentative skills (2026) · cites this paper as Chen et al. [5]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, Chen et al. [5] found that computer-generated feedback could enhance EFL learners’ engagement and improve the quality of written products”
From Evaluating the impact of using ChatGPT on EFL students’ argumentative skills (2026) · cites this paper as Chen et al. [5]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some studies found stronger effects among lower-proficiency learners (Wei et al., 2023; Shen et al., 2023)”
From The Use of Artificial Intelligence Tools in English Academic Writing among University Students: A Scoping Review (2025) · cites this paper as Shen et al., 2023
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 31,709
Qualitative Inquiry and Research Design: Choosing Among Five Approaches
Lewis, S. (2015) · Health Promotion Practice · doi:10.1177/1524839915580941 · Cited by 22,105
Cohen, L., Manion, L., & Morrison, K. (2007) · Routledge eBooks · doi:10.4324/9780203029053 · Cited by 20,105
(2017) · Cited by 16,319
Complex Systems and Applied Linguistics
Cameron, L., & Larsen-Freeman, D. (2007) · International Journal of Applied Linguistics · doi:10.1111/j.1473-4192.2007.00148.x · Cited by 1,317
Error feedback in L2 writing classes
Ferris, D., & Roberts, B. (2001) · Journal of Second Language Writing · doi:10.1016/s1060-3743(01)00039-x · Cited by 981
Complexity, accuracy and fluency in second language acquisition
Housen, A., & Kuiken, F. (2009) · Applied Linguistics · doi:10.1093/applin/amp048 · Cited by 863 · cited 1×
“It might evolve slowly since it is more dependent on the psycholinguistic process when applying L2 knowledge (Housen and Kuiken, 2009).”
BackgroundLarsen-Freeman, D. (2006) · Applied Linguistics · doi:10.1093/applin/aml029 · Cited by 805
LU, X. (2011) · TESOL Quarterly · doi:10.5054/tq.2011.240859 · Cited by 744
Research works printed but not linked: 1
- Wolfe-Quintero K. Inagaki S. Kim H.-Y. (1998). Second language development in writing: Measures of fluency, accuracy, & complexity. Honolulu: Second Language Teaching & Curriculum Center, University of Hawaii at Manoa.
Cited by
Cai, L., Msafiri, M. M., & Kangwa, D. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-13112-0 · Cited by 104
The Impact of AI on the Personal and Collaborative Learning Environments in Higher Education
Msambwa, M. M., Wen, Z., & Daniel, K. (2025) · European Journal of Education · doi:10.1111/ejed.12909 · Cited by 76
Daniel, K., Msambwa, M. M., & Wen, Z. (2025) · European Journal of Education · doi:10.1111/ejed.70036 · Cited by 71
A systematic review of AI-based feedback in educational settings
Durak, H. Y., & Onan, A. (2025) · Journal of Computational Social Science · doi:10.1007/s42001-025-00428-1 · Cited by 89
Min, H., Abbasi, B. N., & Jing, H. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-05817-5 · Cited by 76
Ding, L., Zou, D., & Kohnke, L. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13775-3 · Cited by 79
Kangwa, D., Msafiri, M. M., & Fute, A. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70109 · Cited by 22
Luo, L., & Yusuf, A. (2026) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2025.2548919 · Cited by 21
Román-Acosta, D. (2024) · LatIA · doi:10.62486/latia2024110 · Cited by 34
Kangwa, D., Msafiri, M. M., & Fute, A. (2025) · AI and Ethics · doi:10.1007/s43681-025-00689-6 · Cited by 17
“…concerns [85] Applications in personalized learning and assessment Ethical use in content creation High intention to use for personalized learning [89] Immediate scores and diagnostic information aid writing improvement Ethical use in writing tasks High motivation due to immediate feedback [22]…”
The Future of Artificial Intelligence in Interactive Learning: Trends, Challenges, Opportunities
Alam, T. H. I., & Windiarti, I. S. (2025) · Mechanical Engineering, Science and Technology International Conference (MEST) · doi:10.3390/engproc2025084087 · Cited by 16
Dong, H. (2025) · Psychology in the Schools · doi:10.1002/pits.23521 · Cited by 13
A Systematic Review on AI Technologies in English as a Foreign Language Education
Toscu, S. (2025) · Sakarya University Journal of Education · doi:10.19126/suje.1623401 · Cited by 14
Du, C., & Yang, Y. (2025) · System · doi:10.1016/j.system.2025.103828 · Cited by 17
Wang, Y., Zhang, Z. J., & Zhou, H. (2026) · Research Synthesis in Applied Linguistics · doi:10.1080/29984475.2026.2647961 · Cited by 15
Bo, Z., Pek, L. S., Jian, L., Cong, W., & Nan, L. T. (2025) · Language Teaching Research Quarterly · doi:10.32038/ltrq.2025.53.05 · Cited by 6
How AI shapes self-regulated learning in foreign language education
Hou, F., & Zhou, C. (2025) · Discover Computing · doi:10.1007/s10791-025-09801-0 · Cited by 21
Optimizing Writing Skills in Children Using a Real-Time Feedback System Based on Machine Learning
Villegas-Ch, W. E., García-Ortiz, J., & Sánchez-Viteri, S. (2024) · IEEE Access · doi:10.1109/access.2024.3492974 · Cited by 8
Askarkyzy, S., & Жунусбекова, А. (2024) · Pedagogy and Psychology · doi:10.51889/2960-1649.2024.61.4.008 · Cited by 9
Haddadian, G., Kim, M. K., & Haddadian, N. (2025) · Research Synthesis in Applied Linguistics · doi:10.1080/29984475.2025.2598266 · Cited by 5
Mapping the landscape of AI-driven feedback in education: a scoping review
Lipnevich, A. A., Taranto, C. A., DeLuca, C., Nukpetsi, E., Rickey, N., Hopfenbeck, T. N., Carter, E., & McGrane, J. A. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1799346 · Cited by 5
Ndruru, F., Cahyono, B. Y., Rovikasari, M., & Mulati, D. F. (2025) · Journal of Information Technology Education Research · doi:10.28945/5609 · Cited by 11
Ai Generated vs. Peer Feedback in Esl Writing:Effects on Writing Skill, Self-Efficacy, And EnjoymentPreprint
Fang, Y. G., Tan, Y., Zuo, C., & Boubaker, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5115438 · Cited by 6
Evaluating the impact of using ChatGPT on EFL students’ argumentative skills
Mai, H. T. (2026) · Discover Education · doi:10.1007/s44217-026-01153-1 · Cited by 2
Alshehri, O. A. O., Lamouchi, A., Abdellatif, M. S., Al-Dosari, M. N. A., Mekheimer, M., & Nemt-allah, M. A. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1830103 · Cited by 2
Gou, R., Yang, X., Chen, X., & Wang, H. (2025) · Reading and Writing · doi:10.1007/s11145-025-10719-8 · Cited by 4
Chen, Z., & Goh, P. S. C. (2026) · Cogent Education · doi:10.1080/2331186x.2026.2665495 · Cited by 1
Incorporating Artificial Intelligence as a Tool in Teaching Writing Skills
Alhiti, S. K. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-0771-8.ch003 · Cited by 2
Karjo, C. H. (2025) · International Conference on Education Technology and Computer · doi:10.1109/icetc66579.2025.11387276
The Impact of AI-Assisted Learning on the Agency of Foreign Language Learners: A Meta-Analysis
Zai, F., & Zhou, X. (2026) · Behavioral Sciences · doi:10.3390/bs16030379 · Cited by 6
Enhancing Writing Through RoBERTa-Based Transfer Learning
Wang, Y. (2026) · International Journal of Information Technologies and Systems Approach · doi:10.4018/ijitsa.405417
Khan, M. S. H., & Habib, R. B. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101099 · Cited by 3
The influence of GenAI on second language writing instruction: A systematic review
Wang, Z., Lei, H., & Chen, K. (2026) · System · doi:10.1016/j.system.2026.104081 · Cited by 9
Exploring EFL Learners’ Experiences with Automated and AI-Based Feedback Tools in Writing Classrooms
As'ary, M. A., & Syamsuddin, M. R. (2026) · DEIKTIS Jurnal Pendidikan Bahasa dan Sastra · doi:10.53769/deiktis.v6i2.3663
Literature Review: Analysing the Factors Affecting College Students’ Digital Language Tests
Zhang, J., & Razak, R. A. (2026) · Africa Education Review · doi:10.1080/18146627.2026.2659587
Learner perceptions of AI-generated feedback on L2 speaking in the ESP context
Yolum, M., & Atay, D. (2026) · English for Specific Purposes · doi:10.1016/j.esp.2026.06.002 · Cited by 2
Cai, B., Fu, H. C., He, Z., Wang, Y., & Song, Y. (2026) · Computer Assisted Language Learning · doi:10.1080/09588221.2026.2685747 · Cited by 1
Jia, Q., & Zhang, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7077978
PEUMO: An AI-based writing assessment and support tool for engineering report writing in Spanish
Canales, C. C., & René Venegas, V. (2026) · Assessing Writing · doi:10.1016/j.asw.2026.101106
Zhang, J., & Wang, S. (2026) · European Journal of Education · doi:10.1111/ejed.70818
Fang, Y., Tan, Y., Zuo, C., & Boubaker, A. (2026) · International Conference on Computer Science and Technologies in Education (CSTE) · doi:10.1109/cste69562.2026.11649771
The impact of AI-Assisted writing tools on L2 academic writing performance in an Indonesian context
Yasnalia, S., & Ariyanto, N. S. (2025) · HAMKA INSIGHT · doi:10.51574/hamka.v5i2.129 · Cited by 1
Metadata
- Authors
- Shen, C., Shi, P., Guo, J., Xu, S., & Tian, J.
- Affiliations
- Chang'an University (CN)
- Xi'an Jiaotong University (CN)
- Xi'an Jiaotong University City College (CN)
- Air Force Engineering University (CN)
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2023-10-23
- Volume
- 14
- Pages
- 1258286-1258286
- ISSN
- 1664-1078 (ISSN-L)
- PMC
- PMC10701688
- OpenAlex
- W4387878129
- Cited by
- 42
- References
- 54
- Citations by year
- 2026: 17 · 2025: 21 · 2024: 4
- FWCI
- 47.02
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Psychology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
- Social Sciences › Education › English Language Learning and Teaching
- Keywords
- automated writing evaluation, automated feedback, EFL writing, second language writing, revision strategies, feedback uptake, language proficiency, interlanguage development, writing quality, student engagement, student perceptions, error correction, lexical complexity, fluency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
