Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students
Open accessThe DOI matches its Crossref record; 68 of the 71 references it cites were found in a registry or an index; 20 of the 47 sources its claims cite were checked for support.
Abstract
A mixed-methods study of 50 Chinese EFL students randomly assigned to AI-assisted ChatGPT versus traditional instruction, reporting significant improvements in writing skills and motivation.
Overview
CoChat Summary
TL;DR
This mixed-methods study found that 12 weeks of ChatGPT-assisted writing instruction improved Chinese EFL students’ academic writing skills and writing motivation more than traditional instruction. Qualitative interviews with experimental-group students were largely positive but raised concerns about contextual accuracy, over-reliance, and long-term sustainability.
Key findings
- The experimental group outperformed the control group on global writing post-test after controlling for pre-test scores: M = 59.12 vs. 45.18; ANCOVA F(1, 47) = 19.01, p = 0.00, partial eta squared = 0.27; t-test t(48) = 3.79, p < 0.001, Cohen’s d = 0.76.
- Specific writing dimensions favored the experimental group: content F(1, 47) = 12.61, p = 0.00, partial eta squared = 0.19; organization F(1, 47) = 23.16, p = 0.00, partial eta squared = 0.34; language use F(1, 47) = 9.86, p = 0.01, partial eta squared = 0.11.
- Writing motivation was significantly higher in the experimental group: M = 20.06 vs. 18.21; ANCOVA F(1, 47) = 29.62, p = 0.00, partial eta squared = 0.38; t-test t(48) = 3.34, p = 0.001, Cohen’s d = 0.52.
- Qualitative themes included positive perceptions of innovation and assistance, writing skill enhancements, motivation and engagement gains, recognized advantages such as accessibility and personalized feedback, acknowledged challenges including contextual accuracy and over-reliance, and concerns about long-term impact and sustainability.
Methods
- Mixed-methods design with 50 Chinese EFL undergraduates at a national university, matched for proficiency and randomly assigned to experimental (ChatGPT-assisted instruction) or control (traditional teacher-led instruction) groups; intervention lasted 12 weeks, with AI sessions twice weekly for approximately 60 minutes.
- Pre-test and post-test IELTS academic writing tasks 1 and 2 were scored with IELTS band descriptors by two raters (inter-rater reliability r = 0.88); writing motivation was measured with a 7-item Writing Motivation Scale on a 5-point Likert scale (Cronbach’s α = 0.86); semi-structured interviews were conducted with 9 experimental-group students for about 30 minutes each and analyzed thematically.
- Quantitative analysis used one-way ANCOVA controlling for pre-test scores, independent samples t-tests, and effect sizes.
Limitations
- Stated by the paper: generalizability is constrained by the specific sample of Chinese EFL students; the sample size of 50 is relatively small; the intervention duration was relatively short; possible contamination between experimental and control groups; no long-term follow-up to assess sustainability; and potential unintended plagiarism or over-reliance on AI-generated content despite ethical guidance.
- Evident from the text: writing motivation was measured only by self-report questionnaire, and qualitative interviews were conducted only with experimental-group students, so control-group perceptions and experiences were not captured.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
14 Valid · 3 Not valid · 17 Not checked · 81 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Importantly, feedback is recognized as a critical component in enhancing writing performance and motivation in the context of language learning (Bakla, 2020; Liu et al., 2022; Zhang and Zou, 2023).”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Literature review > Theoretical framework
What the sources say
- Supportedcites Bakla, 2020 source
The Introduction describes feedback as an essential part of second-language writing instruction and explains how it can facilitate learning. The Results from the Essay-Revision Task report successful revisions following feedback, while the Discussion says digital feedback motivated students to work on their writing. These passages support the broad claim, though they do not establish that one feedback mode improves performance more than another. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites Liu et al., 2022 source
- Not checked — no full text of the sourcecites Zhang and Zou, 2023 source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This heightened engagement, facilitated by the AI tool’s responsiveness, contributes to the observed improvements in writing skills and motivation (Liu et al., 2021; Utami and Winarni, 2023).”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Literature review > Theoretical framework
What the sources say
- Not checked — no full text of the sourcecites Liu et al., 2021 source
- Partly supportedcites Utami and Winarni, 2023 sourceExperimental — accuracy not yet measured
The Discussion reports that students perceived improved motivation, and the writing-skills findings report perceived benefits, with observed writing-quality improvement limited to creativity of ideas (section labeled “3.74”; page unavailable). The paper does not establish that the tool’s responsiveness heightened engagement or that such engagement caused improvements in skills and motivation. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The identified contextual accuracy concerns resonate with previous studies highlighting the need for careful consideration when incorporating AI-generated feedback (Utami and Winarni, 2023).”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Discussion
What the sources say
- Partly supportedcites Utami and Winarni, 2023 sourceExperimental — accuracy not yet measured
In “Students’ perception of usefulness of artificial intelligence technology utilization” (page unavailable), a participant says users must decide whether to accept AI-offered sentences and rearrange drafts so they remain logical. This supports careful consideration of AI writing suggestions, but the paper does not identify contextual accuracy problems in AI-generated feedback specifically; it says participants did not use the tools for revising and editing. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The intervention’s effectiveness in enhancing organization and coherence within their essays demonstrates the potential of AI-assisted tools in scaffolding the writing process (Zhao, 2022; Barrot, 2023).”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Discussion
What the sources say
- Partly supportedcites Zhao, 2022 sourceExperimental — accuracy not yet measured
The Introduction (p. 2) says Wordtune supports EFL writers during the writing process, and the Conclusion and Limitations (p. 6) says it can help improve writing quality. But this tech review does not report an intervention or demonstrate improvements in essay organization and coherence. (LLM, conf=0.87)
- Not checked — no full text of the sourcecites Barrot, 2023 source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Significant differences were observed in writing motivation between the two groups, as presented in Table 7 [F(1, 47) = 29.62, p = 0.00, partial eta squared = 0.38].”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Results > Quantitative results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
44 more claims
Create a free account to see all 115 claims and every source verdict.
- It may state two separate claims as one. 26
- Probably not a substantial claim. 42
- Borderline: one of the tests for a substantial claim is unclear. 2
- Not yet classified. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These findings align with studies from other EFL settings [31] where instructors appreciate AI's ability to streamline lesson planning and increase productivity.”
From Teachers' Use of Generative AI in Jordanian Universities: Practices and Perceptions (2025) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prior research suggests that technology-assisted learning environments can enhance learners' confidence when tools are perceived as facilitators of understanding rather than substitutes for thinking (Heo et al., 2021; Ma and Lee, 2021; Song and Song, 2023).”
From Cognitive offloading through digital tools and its relationship with critical thinking, task persistence, and learning depth (2026) · cites this paper as Song and Song, 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Students were found to engage actively with AI-generated suggestions, selectively modifying or rejecting content, which indicated maintained agency and thoughtful decision making.”
From AI-Integrated Scaffolding to Enhance Agency and Creativity in K-12 English Language Learners: A Systematic Review (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Students gained confidence in using ChatGPT for IELTS tasks.Evidence of growing ownership and self-direction, though creativity limited by the assessment format.”
From AI-Integrated Scaffolding to Enhance Agency and Creativity in K-12 English Language Learners: A Systematic Review (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Students gained confidence in using ChatGPT for IELTS tasks.Evidence of growing ownership and self-direction, though creativity limited by the assessment format.”
From AI-Integrated Scaffolding to Enhance Agency and Creativity in K-12 English Language Learners: A Systematic Review (2025) · cites this paper as [7]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Statistics that don’t add up
Each reported p-value recomputed from its test statistic and degrees of freedom.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“t(48) = 3.01, p = 0.003”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Results > Quantitative results
Reported p = .003 · recomputed p = .004
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“t(48) = 3.34, p = 0.001”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Results > Quantitative results
Reported p = .001 · recomputed p = .002
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Doesn’t add upt(48) = 3.01reported p = .003 · recomputed p = .004
“t(48) = 3.01, p = .003”
- Doesn’t add upt(48) = 3.34reported p = .001 · recomputed p = .002
“t(48) = 3.34, p = .001”
- Consistentt(48) = 3.79reported p < .001 · recomputed p < .001
“t(48) = 3.79, p < .001”
- Consistentt(48) = 4.12reported p < .001 · recomputed p < .001
“t(48) = 4.12, p < .001”
- Consistentt(48) = 4.34reported p < .001 · recomputed p < .001
“t(48) = 4.34, p < .001”
5 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Method > Procedure
“Table 1 indicates the summary of the intervention details for both groups.”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Results > Quantitative results
“Table 1 presents descriptive statistics for the pre-and post-test scores in both the experimental and control groups.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Method > Instruments
“Semi-structured interviews (see Appendix C) were conducted with nine EFL students from the experimental classroom group”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Results > Qualitative results
“Likewise, thinking about the long-lasting effects, participant S14 said”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Results > Quantitative results
“Furthermore, the ANCOVA results for the two groups’ language use are depicted in Table 6.”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Results > Quantitative results
“Table 5 reveals significant distinctions between the two groups in terms of language use [F(1, 47) = 9.86, p = 0.01, partial eta squared = 0.11].”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Results > Quantitative results
“Also, the ANCOVA results for the two groups’ writing organization are displayed in Table 4.”
§ Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students > Results > Quantitative results
“Significant disparities between the two groups in terms of writing organization were observed, as indicated by the results presented in Table 5”
Citations
Cites
(1998) · Cited by 13,501
Becoming a Self-Regulated Learner: An Overview
Zimmerman, B. J. (2002) · Theory Into Practice · doi:10.1207/s15430421tip4102_2 · Cited by 10,197 · cited 1×
“The AI tool acts as a facilitator, guiding learners to internalize effective writing strategies and enabling them to complete tasks independently (Zimmerman, 2002).”
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242
(2016) · Open Research Online (The Open University) · Cited by 1,410
Evolution and Revolution in Artificial Intelligence in Education
Roll, I., & Wylie, R. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-016-0110-3 · Cited by 1,229 · cited 1×
“…of technology in English language classrooms is widely recognized as a means to overcome certain obstacles in language learning processes (Roll and Wylie, 2016; Knox, 2020), particularly in writing tasks where time constraints often arise (Stapleton and Radia, 2010; Kessler, 2020; Rahimi…”
Fredricks, J. A., Filsecker, M., & Lawson, M. A. (2016) · Learning and Instruction · doi:10.1016/j.learninstruc.2016.02.002 · Cited by 851 · cited 1×
“Engagement, as a crucial aspect of the learning process, is influenced by the dynamic interaction between learners and their learning environment (Fredricks et al., 2016).”
Can artificial intelligence help for scientific writing?
Salvagno, M., Taccone, F. S., & Gerli, A. G. (2023) · Critical Care · doi:10.1186/s13054-023-04380-2 · Cited by 827 · cited 1×
“Additionally, learners can expedite their writing development as AI tools help them identify and rectify grammatical and lexical errors, while also suggesting alternative sentence structures to enhance the overall writing quality and structure (Zhao, 2022; Chen, 2023; Salvagno et al., 2023).”
Using ChatGPT for second language writing: pitfalls and potentials
Barrot, J. S. (2023) · Assessing Writing · doi:10.1016/j.asw.2023.100745 · Cited by 770 · cited 4×
“This is consistent with studies by Barrot (2023) and Zhao (2022), which emphasize how immediate feedback can significantly contribute to enhanced writing proficiency.”
InfluentialImpact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation
Yan, D. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11742-4 · Cited by 713 · cited 8×
“This includes the use of advanced artificial intelligence (AI)-based computer and mobile programs, which offer interactive and personalized tools for honing writing abilities as well as enhanced motivation (Jiang, 2022; Meunier et al., 2022; Yan, 2023).”
InfluentialHuang, A. Y. Q., Lu, O. H. T., & Yang, S. J. H. (2023) · Computers & Education · doi:10.1016/j.compedu.2022.104684 · Cited by 605
Research works printed but not linked: 2
- Pallant, 2013, SPSS survival manual
- Vygotsky, 1984, The problem of age, Pedagogical
Cited by
Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13544 · Cited by 805
Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies
Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2025) · Computers & Education · doi:10.1016/j.compedu.2024.105224 · Cited by 531
Using AI-driven chatbots to foster Chinese EFL students’ academic engagement: An intervention study
Wang, Y., & Xue, L. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108353 · Cited by 328
Wang, J., & Fan, W. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04787-y · Cited by 322
“The meta-analysis results show that ChatGPT effectively enhances student learning performance across different types of courses, as has been shown by prior studies (Ba et al. 2024; Li, 2023; Song and Song, 2023).”
RetractedHalkiopoulos, C., & Gkintoni, E. (2024) · Electronics · doi:10.3390/electronics13183762 · Cited by 328
“Interactivity and personalized feedback have thus become crucial variables for effective language learning [191].”
Metadata
- Authors
- Song, C., & Song, Y.
- Affiliations
- North Minzu University (CN)
- Central South University (CN)
- Corresponding author
- Song, Cuiping
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2023-12-15
- Volume
- 14
- Pages
- 1260843-1260843
- ISSN
- 1664-1078 (ISSN-L)
- PMC
- PMC10754989
- OpenAlex
- W4389793309
- Cited by
- 717
- References
- 70
- Citations by year
- 2026: 297 · 2025: 296 · 2024: 123 · 2023: 1
- FWCI
- 802.96
- Type
- journal article
- Language
- en
- Update notices
- correction (2026-07-22)
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Psychology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Education › English Language Learning and Teaching
- Keywords
- ChatGPT, AI-assisted language learning, academic writing, writing skills, writing motivation, EFL learners, Chinese learners of English, generative AI in education, overreliance on AI, writing instruction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. https://app.cochat.ai/papers/10.3389/fpsyg.2023.1260843
- The paper
- Song, C., & Song, Y. (2023). Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2023.1260843
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
