The DOI matches its Crossref record, with a small difference in title or year; 61 of the 61 references it cites were found in a registry or an index.
Comparing the effects of ChatGPT and automated writing evaluation on students’ writing and ideal L2 writing self
Open accessAbstract
The affordances of ChatGPT in language learning and teaching have gained increasing traction. While studies began to investigate the potential of ChatGPT as a feedback provider, little attention was given to ChatGPT’s potential impact on students’ writing performance and the ideal L2 writing self vis-à-vis the established automated writing evaluation systems (AWE). To address these gaps, a sequential explanatory mixed methods design was adopted. One hundred and fifty second-year university students from three writing classes in a Chinese public university were recruited and randomly divided into a ChatGPT group, an AWE group, and a control group. After an eleven-week intervention, the ANCOVA results showed that while the ChatGPT group scored significantly higher than the AWE group and the control group in post-writing performance as measured by their writing score, in terms of students’ ideal L2 writing self, the ChatGPT group performed significantly lower than the AWE group with a medium effect size. Qualitative analysis of students’ reflection papers revealed students’ (over)reliance on the tool and the accompanying loss of creativity and agency. Pedagogical implications as well as directions for future research are also discussed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“SBERT sentence embeddings can measure semantic distance between initial and revised drafts, helping identify meaning preservation, moderate extension, and obvious deviation [7].”
From Semantic Response Mechanisms of L2 Writing Revision Behavior under Generative AI Feedback (2026) · cites this paper as [7]
Citations
Cites
(1978) · Medical Entomology and Zoology · Cited by 53,488
(2017) · Cited by 16,319
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · cited 1×
“Feedback plays an important role in enhancing learners’ writing ability (Hattie & Timperley, 2007; Hyland & Hyland, 2019).”
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 1×
“These findings consolidated the concern expressed in Kasneci et al. (2023) that the effortlessly generated information could negatively impact students’ critical thinking and problem-solving skills as ChatGPT simplified the process of acquiring information, and by doing so increased students’…”
Markus, H., & Nurius, P. (1986) · American Psychologist · doi:10.1037/0003-066x.41.9.954 · Cited by 5,806
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
Perreault, K. (2010) · Manual Therapy · doi:10.1016/j.math.2010.09.003 · Cited by 4,952
2. The L2 Motivational Self System
Dörnyei, Z. (2009) · Motivation, Language Identity and the L2 Self · doi:10.21832/9781847691293-003 · Cited by 2,052
ChatGPT for Language Teaching and Learning
Kohnke, L., Moorhouse, B. L., & Zou, D. (2023) · RELC Journal · doi:10.1177/00336882231162868 · Cited by 1,371 · cited 2×
“For ChatGPT (OpenAI, 2023), we provided students with the following exemplar prompts during the training session based on Kohnke et al. (2023) and Guo and Wang (2023) and encourage students to use all of them for every revision task: (1)You are an expert writer, please point out and correct all the…”
Walter, Y. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00448-3 · Cited by 1,113 · cited 1×
“Meanwhile, prompt engineering has gradually been regarded as an important skill that students need to be equipped with (Miao & Holmes, 2023; Tseng & Warschauer, 2023; Walter, 2024).”
Research works printed but not linked: 5
- Burstein J. Chodorow M. & Leacock C. (2003). Criterion online essay evaluation: An application for automated evaluation of student essays [Paper presentation]. Proceedings of the Fifteenth Annual Conference on Innovative Applications of Artificial Intelligence Acapulco: Mexico.
- 10.64152/10125/73570 · DOI
- Saldaña, J. (2016). The coding manual for qualitative researchers (3rd ed.). Sage Publications.
- Shi, J., & Shuai, P. (2010). English writing: From sentence to paragraph to theme. Foreign Language Teaching and Research Press.
- Stockwell, G., & Wang, Y. (2024b). Motivation and autonomy. In R. Hampel & U. Stickler (Eds.), The Bloomsbury Handbook of Language Learning and Technology (pp. 144–156). Bloomsbury Academic.
Cited by
Sorted by citations
Generative AI and L2 Written Feedback Studies: A Scoping Review
Crosthwaite, P., & Sun, S. (2026) · RELC Journal · doi:10.1177/00336882251386530 · Cited by 50
Ding, L., Zou, D., & Kohnke, L. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13775-3 · Cited by 59
Pawlak, M., Kruk, M., Shirvan, M. E., Taherian, T., & Karimpour, S. (2025) · Computer Assisted Language Learning · doi:10.1080/09588221.2025.2498094 · Cited by 29
Generative AI vs. Teachers: Feedback Quality, Feedback Uptake, and Revision
Yu, H., & Xie, Q. (2025) · Language Teaching Research Quarterly · doi:10.32038/ltrq.2025.47.07 · Cited by 22
Teng, L. S., Deng, X., & Yang, J. (2026) · System · doi:10.1016/j.system.2026.103970 · Cited by 16
Metadata
- Authors
- Shi, H., Chai, C. S., Zhou, S., & Aubrey, S.
- Affiliations
- Chinese University of Hong Kong (HK)
- Corresponding author
- Shi, Huawei
- Venue
- Computer Assisted Language Learning
- Publisher
- Informa UK Limited
- Published
- 2025-02-03
- Volume
- 39
- Issue
- 4
- Pages
- 1148-1175
- ISSN
- 0958-8221 (ISSN-L)
- 1744-3210
- OpenAlex
- W4407108474
- Cited by
- 109
- References
- 61
- Citations by year
- 2026: 87 · 2025: 22
- FWCI
- 387.04
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Computer Assisted Language Learning (publishedVersion, cc-by-nc-nd)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Education › English Language Learning and Teaching
- Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
- Keywords
- ChatGPT, automated writing evaluation, second language writing, writing performance, AI-generated feedback, student creativity, student agency, university students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
