CoChat
Sign Up
Cited by 90Artificial Intelligence in EducationEFL/ESL Teaching and LearningEnglish Language Learning and Teaching

The Role of AI-Assisted Learning in Academic Writing: A Mixed-Methods Study on Chinese as a Second Language Students

Open access
Chen, Chen, Gong, Yang (Frank)·2025·Education Sciences
DOI 10.3390/educsci15020141
Open this paper in CoChat — free accountOpen PDF

Abstract

This mixed-methods study examines the role of artificial intelligence (AI)-assisted learning in academic writing for Chinese as a Second Language (CSL) students in a Chinese university context. Fifty international CSL students were randomly assigned to experimental—AI-assisted learning using ChatGPT—and control—traditional learning—groups. Writing samples from the participants were evaluated using established scoring rubrics for Chinese academic writing. Based on pre- and post-test quantitative data and supplementary qualitative interviews with six participants from the experimental group, this study reveals that AI-assisted learning can enhance student outcome by supporting knowledge acquisition, helping to create a supportive learning environment, and increasing student motivation. However, this study also highlights concerns regarding over-reliance on AI, particularly in relation to ethical concerns, technical and networking issues, and the unreliability of AI-generated content. These findings contribute to a nuanced understanding of the impact of AI on CSL learners’ academic writing performance. Finally, we also discuss practical implications for educational stakeholders regarding the integration of AI into language education.

AI snapshot

Written from the paper’s full text.

TL;DR

This mixed-methods study found that ChatGPT-assisted learning improved Chinese as a Second Language (CSL) students’ academic writing scores compared with traditional instruction, while interviews highlighted perceived benefits and concerns. Students described the AI tool as a knowledgeable instructor, supportive learning context, and motivator, but also reported over-reliance, technical issues, and unreliable AI-generated content.

Key findings

  • In the post-test, the AI-assisted group had a higher mean academic writing score (M = 89.74, SD = 9.08) than the traditional group (M = 82.15, SD = 10.23), a significant difference (p < 0.05); pre-test scores did not differ significantly (AI-assisted M = 61.44, SD = 11.06; traditional M = 64.06, SD = 8.42).
  • Repeated measures ANOVA showed significant change in academic writing scores (reported as F3, 87 = 20.79, p < 0.05), and the Bonferroni post hoc test confirmed significant improvement in the experimental group (mean difference = −28.3, p < 0.05).
  • Interview themes described AI-assisted learning as (1) a knowledgeable instructor, (2) a friendly learning context creator, and (3) a passionate learning motivator, with students reporting grammar/vocabulary correction, idea generation, and expression refinement.
  • Concerns included over-reliance on AI and ethical issues, technical/networking obstacles, and unreliable or fabricated AI-generated content, such as nonexistent references and policy terms differing from official documents.

Methods

  • Two-phase mixed-methods design: 50 third-year international undergraduate CSL students at a Chinese university were assigned to an experimental AI-assisted group (n = 25) using ChatGPT or a control traditional-instruction group (n = 25) during a 16-week academic writing course.
  • Quantitative data were pre-test and post-test academic writing samples of about 2000 Chinese characters, scored with established rubrics by two independent HSK raters; inter-rater reliability was reported as 0.86. Analysis used independent samples t-tests and repeated measures ANOVA.
  • Qualitative data came from semi-structured individual interviews with six experimental-group participants, analyzed using thematic analysis based on Boyatzis (1998).

Limitations

  • The paper states the study had a small sample size and calls for future research with a larger, more diverse learner population, follow-up evaluations, and CSL teachers’ perspectives to triangulate findings.
  • The paper also states future work should explore Chinese local LLMs and, in a future phase, incorporate a control group to help draw causal inferences, although the present study already included a control group.
  • Evident from the text: the study was conducted at one university using convenience sampling, the qualitative component included only six interviewees, and the intervention was limited to a 16-week course without long-term follow-up.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/educsci15020141

Cites

Statistics

Nothing to recompute.

Claims the paper makes

0 of 26 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Bonferroni post hoc test confirmed significant improvement in academic writing performance of the experimental group (p<0.05).

    “The Bonferroni post hoc test further confirmed a significant improvement in the academic writing performance of participants in the experimental group (p < 0.05, Table 4).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The pre-test and post-test writing tasks are equivalent in difficulty and can be compared to measure change.
  • Findingown data

    Participants' academic writing scores showed significant change between pre-test and post-test (F3,87=20.79, p<0.05).

    “As illustrated in Table 3, the participants' academic writing scores showed significant change between the pre-test and post-test (F3, 87 = 20.79,p < 0.05) at the probability level in this study.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The pre-test and post-test writing tasks are equivalent in difficulty and can be compared to measure change.
  • Findingown data

    The six interviewees from the experimental group provided positive feedback on the effectiveness of AI-assisted learning in enhancing their academic writing skills.

    “Overall, the six interviewees from the experimental group provided positive feedback on the effectiveness of AI-assisted learning in enhancing their academic writing skills.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The six interviewees are sufficient to capture the range of CSL students' perceptions.
  • Findingown data

    Two of the six interviewees highlighted their use of AI-assisted learning tools to inspire thoughts and refine expression in their academic writing.

    “Two of the six interviewees highlighted their use of AI-assisted learning tools to inspire thoughts and refine expression in their academic writing.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The six interviewees are sufficient to capture the range of CSL students' perceptions.
  • Findingown data

    Four of the six interviewees reported encountering technical or networking obstacles while using the AI-assisted learning tool.

    “Four of the six interviewees reported encountering technical or networking obstacles while using the AI-assisted learning tool.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 35
  • Classified as probably not a substantial claim. 32

  • May combine two claims.

    Findingown data

    Concerns regarding overreliance on AI include ethical concerns, technical and networking issues, and the unreliability of AI-generated content.

    “However, this study also highlights concerns regarding overreliance on AI, particularly in relation to ethical concerns, technical and networking issues, and the unreliability of AI-generated content.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The six interviewees are sufficient to capture the range of CSL students' perceptions.

  • May combine two claims.

    Findingown data

    Participants saw the AI-assisted learning tool as playing three main roles in their academic writing: a knowledgeable instructor; a friendly learning context creator; and a passionate learning motivator.

    “Through a thematic analysis of the interview data, it was found that participants saw the AI-assisted learning tool as playing three main roles in their academic writing: (1) a knowledgeable instructor; (2) a friendly learning context creator; and (3) a passionate learning motivator.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The six interviewees are sufficient to capture the range of CSL students' perceptions.

  • May combine two claims.

    Findingown data

    ChatGPT offered two main types of feedback to improve students' academic writing: identifying and correcting vocabulary and grammatical errors; and inspiring thoughts and refining expression.

    “Specifically, their responses indicate that ChatGPT offered two main types of feedback to improve students' academic writing: (1) identifying and correcting vocabulary and grammatical errors; and (2) inspiring thoughts and refining expression.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The six interviewees are sufficient to capture the range of CSL students' perceptions.

  • May combine two claims.

    Findingown data

    Interviewees reported using ChatGPT to search for commonly-used phrases, complete a sentence, and look up a specific word.

    “Interviewees also reported that they used ChatGPT “to search for commonly-used phrases” (David, Int.3; Fiona, Interview), “to complete a sentence” (Ava, Interview), and “to look up a specific word” (Bella, Int.2; David, Interview).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The six interviewees are sufficient to capture the range of CSL students' perceptions.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    David found that some references provided by ChatGPT were nonexistent.

    “For instance, David found that some references provided by ChatGPT were nonexistent.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Ava compared ChatGPT with an electronic dictionary, noting that access to ChatGPT remained an obstacle for her.

    “Ava compared ChatGPT with an electronic dictionary, noting that access to ChatGPT remained an obstacle for her.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The results identified challenges and concerns regarding the use of AI-assisted learning, including students' over-reliance on AI, ethical concerns, technical and networking issues, and the potential unreliability of AI-generated content.

    “However, the results also identified challenges and concerns regarding the use of AI-assisted learning, including students' over-reliance on AI, ethical concerns, technical and networking issues, and the potential unreliability of AI-generated content.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Most interviewees felt that the AI-assisted learning tool created a low-pressure learning environment, allowing them to interact without feeling self-conscious or anxious.

    “Most interviewees expressed the feeling that the AI-assisted learning tool created a low-pressure learning environment, allowing them to interact with the tool without feeling self-conscious or anxious.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Participants noted instances where AI tools provided inaccurate or unverified information, raising concerns about the potential negative impact on their writing.

    “Participants noted instances where AI tools provided inaccurate or unverified information, raising concerns about the potential negative impact on their writing (David, Interview; Fiona, Interview).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    AI tools in this study inspired ideas, generated grammatically accurate sentences, and adjusted language to meet academic style expectations, ultimately improving students' performance in academic writing tasks.

    “AI tools in this study inspired ideas, generated grammatically accurate sentences, and adjusted language to meet academic style expectations, ultimately improving students' performance in academic writing tasks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    David and Eric mentioned networking issues, as both had experienced unstable connections when they needed ChatGPT for academic writing.

    “David and Eric mentioned networking issues, as both had experienced unstable connections when they needed ChatGPT for academic writing.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Students used the AI tool to organize their writing, search for phrases, complete sentences, and look up specific words.

    “Students also used the AI tool to organize their writing, search for phrases, complete sentences, and look up specific words.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    AI tools allowed students to learn at their own pace, with content tailored to their needs.

    “AI tools allowed students to learn at their own pace, with content tailored to their needs (David, Interview).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Despite concerns, students continued to depend heavily on AI tools, regardless of their language proficiency or performance levels, raising questions about maintaining academic integrity.

    “Despite these concerns, students continued to depend heavily on AI tools, regardless of their language proficiency or performance levels, raising questions about maintaining academic integrity.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Two interviewees expressed concern that the AI tool might generate unverified or even fabricated content in response to their requests.

    “Two interviewees expressed concern that the AI tool might generate unverified or even fabricated content in response to their requests.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    All six interviewees reported concerns about their own over-reliance on the AI-assisted learning tools in academic writing, as well as the associated ethical issues.

    “All six interviewees reported concerns about their own over-reliance on the AI-assisted learning tools in academic writing, as well as the associated ethical issues.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In this study, AI-assisted learning offered a flexible, supportive environment that students found engaging and motivating.

    “In this study, AI-assisted learning offered a flexible, supportive environment that students found engaging and motivating.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The participants reported positive learning experiences, noting that AI-assisted tools served as a knowledgeable instructor, a creator of a supportive learning environment, and a passionate motivator.

    “The participants reported positive learning experiences, noting that AI-assisted tools served as a knowledgeable instructor, a creator of a supportive learning environment, and a passionate motivator.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The participants' self-reported perceptions and experiences accurately reflect their actual learning processes and outcomes.

  • May combine two claims.

    Findingown data

    The AI tool provided feedback that effectively addressed the participants' writing needs, leading them to perceive the tool as a knowledgeable instructor.

    “In this study, the AI tool provided feedback that effectively addressed the participants' writing needs, which led them to perceive the tool as a knowledgeable instructor (Ava, Interview).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Two independent raters evaluated each writing sample at pre-test and post-test stages.

    “To ensure objectivity in the scoring process, two independent raters evaluated each writing sample at the pre-test and post-test stages.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Social interactions play a crucial role in the language learning process.

    “According to sociocultural theory, social interactions play a crucial role in the language learning process (Luan et al., 2023;Pica et al., 1991).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Luan et al., 2023

      Not checked in the free run.

    • cites Pica et al., 1991

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    AI-assisted learning can incorporate automated feedback on various aspects of writing, including effectiveness, content organization, coherence, grammar, and vocabulary, facilitating students' writing performance.

    “AI-assisted learning can incorporate automated feedback on various aspects of writing, including effectiveness, content organization, coherence, grammar, and vocabulary, facilitating students' writing performance (Song & Song, 2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Song & Song, 2023

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Feedback and interactive experiences provide important scaffolding for students' development of FL/SL skills, improving their experience of and performance in writing tasks.

    “Feedback and interactive experiences provide important scaffolding for students' development of FL/SL skills, thus improving their experience of and performance in writing tasks (Banister, 2023;Lundstrom & Baker, 2009).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Banister, 2023

      Not checked in the free run.

    • cites Lundstrom & Baker, 2009 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Academic writing is a complex, demanding, and stressful task for students.

    “Academic writing is a complex, demanding, and stressful task for students (Yu & Liu, 2021)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yu & Liu, 2021

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Translations of interview excerpts were cross-checked by an independent reviewer proficient in both Mandarin and English.

    “To ensure accuracy and reliability, the translations were initially completed by the first author, a native Mandarin speaker, with careful attention to cultural and contextual differences.These translations were then cross-checked by an independent reviewer proficient in both Mandarin and English.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The quantity and quality of instructions provided to control students were designed to be comparable to those provided to the experimental group.

    “The quantity and quality of instructions provided to these students were designed to be comparable to those provided to their counterparts in the experimental group.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    AI-assisted tools can be understood as a viable substitute for human interaction partners when direct engagement is not feasible.

    “In this context, AI-assisted tools can be understood as a viable substitute for human interaction partners when direct engagement is not feasible (Fathi et al., 2024).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Fathi et al., 2024 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    AI-assisted learning can enhance student outcome by supporting knowledge acquisition, helping to create a supportive learning environment, and increasing student motivation.

    “Based on pre-and post-test quantitative data and supplementary qualitative interviews with six participants from the experimental group, this study reveals that AI-assisted learning can enhance student outcome by supporting knowledge acquisition, helping to create a supportive learning environment, and increasing student motivation.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The pre-test and post-test writing tasks are equivalent in difficulty and can be compared to measure change.
    • Rests on an assumption: The control group participants did not use AI-assisted tools during the intervention.
    • Rests on an assumption: The 10-week intervention period is sufficient for AI-assisted learning to affect academic writing.
    • Rests on an assumption: The six interviewees are sufficient to capture the range of CSL students' perceptions.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning

    There remains a notable gap in the literature concerning AI tools' application specifically to CSL students.

    “Despite the growing body of research on the effectiveness of AI tools in enhancing FL/SL learners' academic writing skills, there remains a notable gap in the literature concerning their application specifically to CSL students.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Thematic analysis coding was completed by two independent coders.

    “The coding process was completed by two independent coders, both of whom were experienced CSL teachers and HSK writing raters, to ensure the reliability of the data analysis.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The two raters conducted a double-blind evaluation of the writing samples.

    “The two raters then conducted a double-blind evaluation of the writing samples with reference to the established scoring rubrics.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Fifty international CSL students were randomly assigned to experimental AI-assisted learning using ChatGPT and control traditional learning groups.

    “Fifty international CSL students were randomly assigned to experimental-AI-assisted learning using ChatGPT-and control-traditional learninggroups.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    AI holds the potential to enhance student-teacher interaction by offering timely feedback and facilitating personalized communication, thereby addressing some limitations of traditional instructional methods.

    “AI holds the potential to enhance studentteacher interaction by offering timely feedback and facilitating personalized communication, thereby addressing some of the limitations often associated with traditional instructional methods.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    AI-assisted learning could improve students' learning autonomy and motivation.

    “Song and Song (2023) investigated a group of Chinese undergraduate English as a foreign language (EFL) learners and found that AI-assisted learning could improve students' learning autonomy and motivation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Song & Song, 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    ChatGPT's multilingual and cross-cultural capabilities can be suitable for international CSL students to provide feedback addressing both structural and stylistic aspects of writing.

    “ChatGPT's multilingual and cross-cultural capabilities can be suitable for them to provide feedback addressing both the structural and stylistic aspects of writing.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Overall, the various roles that ChatGPT played in the participants' development of academic writing skills enhanced their learning motivation and their engagement in writing activities.

    “Overall, the various roles that ChatGPT played in the participants' development of academic writing skills enhanced their learning motivation and their engagement in writing activities.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Over-reliance on AI support can hinder the development of independent critical thinking and creativity.

    “A major issue was the over-reliance on AI support in writing tasks, which can hinder the development of independent critical thinking and creativity, as noted by Utami and Winarni (2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Utami and Winarni (2023)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In many Chinese universities, CSL students face intense peer competition and external pressures, which can discourage open communication and active participation in class.

    “In many Chinese universities, CSL students face intense peer competition and external pressures (Gao et al., 2014;Gong et al., 2021), which can discourage open communication and active participation in class (Fiona, Interview).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Gao et al., 2014

      Not checked in the free run.

    • cites Gong et al., 2021

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    These perceptions highlight the contribution of AI-assisted tools to enhancing learners' academic writing motivation and skills.

    “These perceptions highlight the contribution of AI-assisted tools to enhancing learners' academic writing motivation and skills.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    This finding supports Zhao's (2023) report that AI tools are often more effective than traditional methods for meeting students' needs in academic writing.

    “This finding supports Zhao's (2023) report that AI tools are often more effective than traditional methods for meeting students' needs in academic writing.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhao (2023) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The study's findings align with previous studies (Liu et al., 2021; Song & Song, 2023) that AI can enhance FL/SL learners' writing abilities.

    “These findings align with previous studies (e.g., Liu et al., 2021;Song & Song, 2023), which highlight AI's potential to enhance FL/SL learners' writing abilities.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Liu et al., 2021

      Not checked in the free run.

    • cites Song & Song, 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Students using AI tools tend to demonstrate higher engagement in writing tasks and produce better writing outcomes compared to peers using traditional paper-based tools.

    “Empirical studies have suggested that, compared to their peers relying on traditional paper-based tools, students using AI tools tend to demonstrate higher engagement in writing tasks and produce better writing outcomes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    This personalized support enabled students to receive specific feedback on their writing, helping them address individual challenges effectively.

    “This personalized support enabled students to receive specific feedback on their writing, helping them address individual challenges effectively, as noted by Song and Song (2023).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Song and Song (2023)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    AI writing tools offered various language support functions, such as sentence or text generation, which could improve students' writing performance.

    “Similarly, Dale and Viethen (2021) reported that AI writing tools offered various language support functions, such as sentence or text generation, which could improve students' writing performance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dale & Viethen, 2021 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    AI not only served as a knowledgeable instructor but also as a motivator, addressing some of the unique challenges faced by CSL students and facilitating more efficient and effective learning.

    “Thus, AI not only served as a knowledgeable instructor but also as a motivator, addressing some of the unique challenges faced by CSL students and facilitating more efficient and effective learning.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    This study's limitations include its small sample size.

    “This study's limitations include its small sample size; future research should examine the potential long-term impact of AI-assisted learning with a larger and more diverse learner population and incorporate follow-up evaluations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    Despite these limitations, the findings of this research demonstrate the value of integrating AI-assisted learning in university language programs.

    “Despite these limitations, the findings of this research demonstrate the value of integrating AI-assisted learning in university language programs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    AI-powered writing evaluation tools effectively enhanced learners' writing quality.

    “Link et al. (2022), Huang and Wilson (2021), and Nunes et al. (2022)consistently emphasized the role of AI-powered writing evaluation tools, identifying that these automated systems effectively enhanced learners' writing quality.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Link et al., 2022 source

      Not checked in the free run.

    • cites Huang & Wilson, 2021

      Not checked in the free run.

    • cites Nunes et al., 2022

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The participants were able to engage more actively in their individual learning processes and had more opportunities to practice and develop their writing skills.

    “Thus, the participants were able to engage more actively in their individual learning processes and had more opportunities to practice and develop their writing skills.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisereasoning

    ChatGPT has been trained on a broad range of academic genres and writing styles, and thus has become suitable for supporting students' academic writing practice.

    “At the same time, ChatGPT has been trained on a broad range of academic genres and writing styles, and thus has become suitable in terms of supporting students' academic writing practice.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Students frequently used AI tools for text translation, which aided them in generating, organizing, and revising their written work.

    “In addition, Godwin-Jones (2019), Urlaub and Dessein (2022), and Zhang and Torres-Hostench (2022) found that students frequently used AI tools for text translation, which aided them in generating, organizing, and revising their written work.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Godwin-Jones, 2019

      Not checked in the free run.

    • cites Urlaub & Dessein, 2022

      Not checked in the free run.

    • cites Zhang & Torres-Hostench, 2022

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    This is particularly valuable given the specific difficulties faced by CSL learners, such as mastering Chinese rhetorical norms and distinguishing between formal and informal language.

    “This is particularly valuable given the specific difficulties faced by CSL learners, such as master-ing Chinese rhetorical norms and distinguishing between formal and informal language.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The AI feedback was particularly helpful for CSL learners, who often struggle with distinguishing between casual and formal expressions, an issue not typically encountered in their native languages.

    “This feedback was particularly helpful for CSL learners, who often struggle with distinguishing between casual and formal expressions, an issue not typically encountered in their native languages.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    The findings offer compelling evidence in favor of integrating AI-assisted learning tools into academic writing.

    “Overall, the findings offer compelling evidence in favor of integrating AI-assisted learning tools into academic writing.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    CSL academic writing, with its distinct vocabulary, sentence structure, and logical coherence, poses unique challenges for learners.

    “CSL academic writing, with its distinct vocabulary, sentence structure, and logical coherence (Shu, 2024), poses unique challenges for learners.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Shu, 2024

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The AI feedback helped students identify and correct errors, improving their writing quality.

    “This feedback helped students identify and correct errors, improving their writing quality.”
    Quote in the paperSubstantial claim
56 of 56

  • The paper relies on this without support.

    The pre-test and post-test writing tasks are equivalent in difficulty and can be compared to measure change.

    “These were administered as course assignments, and were extracted during Weeks 5-6 for the pretest (the mid-term assignment) and Weeks 15-16 (the final assignment) for the post-test, respectively.”

  • The paper relies on this without support.

    The control group participants did not use AI-assisted tools during the intervention.

    “Participants in the control group, meanwhile, continued their traditional teacherinstruction course.They used the same instructional materials as the experimental group.”

  • The paper relies on this without support.

    The 10-week intervention period is sufficient for AI-assisted learning to affect academic writing.

    “This ensured that they had regular exposure to AI-assisted learning throughout the 10-week intervention period.”

  • The paper relies on this without support.

    The six interviewees are sufficient to capture the range of CSL students' perceptions.

    “To gather in-depth qualitative data on the participants' perceptions of the role of AIassisted learning in their academic writing, semi-structured individual interviews were conducted with six participants in the experimental group.”

  • The paper relies on this without support.

    The pre-test and post-test academic writing assessments validly measure CSL students' academic writing ability.

    “The results, drawn from pre-test and post-test assessments as well as six individual interviews, demonstrate that AI-assisted learning significantly improved participants' writing performance.”

  • The paper relies on this without support.

    ChatGPT is representative of AI-assisted learning tools for academic writing.

    “AI-assisted learning tools like ChatGPT can provide greater accessibility compared to teachers in the traditional classroom, without constraints like time, space, and interpersonal relationships.”

  • The paper relies on this without support.

    The improvement in academic writing performance is attributable to the AI-assisted learning intervention rather than to other factors such as practice, maturation, or test familiarity.

    “The results, drawn from pre-test and post-test assessments as well as six individual interviews, demonstrate that AI-assisted learning significantly improved participants' writing performance.”

  • The paper relies on this without support.

    The participants' self-reported perceptions and experiences accurately reflect their actual learning processes and outcomes.

    “The participants reported positive learning experiences, noting that AI-assisted tools served as a knowledgeable instructor, a creator of a supportive learning environment, and a passionate motivator.”

  • The paper relies on this without support.

    The six interviewees' experiences are representative of the broader population of international CSL students in Chinese universities.

    “The findings suggest that AI-assisted learning can significantly enhance students' academic writing skills by providing tailored feedback on language use, grammar, and structure.”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Narrative and table report different repeated-measures ANOVA results

    Major

    The results narrative reports F = 20.79 with degrees of freedom 3 and 87, whereas Table 3 reports F = 3.483 with an effect degree of freedom of 1. They are presented as results of the experimental group's pre-test/post-test analysis.

  • ANOVA F statistic does not match the reported mean squares

    Major

    The reported effect mean square divided by the reported error mean square is about 21.876, not the tabulated F value of 3.483. This makes the reported test statistic for the experimental group's change internally inconsistent.

    Reported 3.483 · recomputed 21.875865

  • ANOVA error sum of squares does not match its mean square

    Minor

    For the sphericity-assumed error row, multiplying the reported degrees of freedom by the reported mean square gives 14.307, not the reported sum of squares of 16.383.

    Reported 16.383 · recomputed 14.307

  • Confidence interval is not centered on its reported mean difference

    Minor

    The midpoint of the reported pre-test-post-test confidence interval is −26.095, rather than the reported mean difference of −28.3. A confidence interval for this pairwise mean difference should be centered on the estimate.

    Reported -28.3 · recomputed -26.095

4 of 4

The citation lists haven’t been fetched yet

References 56 · Cited by 90

Research works printed but not linked: 24

  1. Banister, 2023, Exploring peer feedback processes and peer feedback meta-dialogues with learners of academic and business English, Language Teaching Research, 27, 746, 10.1177/1362168820952222 · DOI
  2. Bhowmik, 2009, L2 writing pedagogy in EFL contexts: An exploration of salient practices in teaching and learning, Journal of Asia TEFL, 6, 351
  3. Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development, Sage Publications.
  4. Chen, 2012, A discussion on the curriculum design for guiding undergraduate students in Chinese writing, Overseas Chinese Education, 1, 24
  5. Fathi, 2022, Examining the impact of flipped classroom on writing complexity, accuracy, and fluency: A case of EFL students, Computer Assisted Language Learning, 35, 1668, 10.1080/09588221.2020.1825097 · DOI
  6. 2019, Riding the digital wilds: Learner autonomy and informal language learning, Language Learning & Technology, 23, 8
  7. Gong, 2021, Developing literacy or focusing on interaction: New Zealand students’ strategic efforts related to Chinese language learning during study abroad in China, System, 98, 102462
  8. Huang, 2023, Trends, research issues and applications of artificial intelligence in language education, Educational Technology & Society, 26, 112
  9. Jiang, 2024, Enhancing critical thinking skills with ChatGPT-powered activities in Chinese language classrooms, International Journal of Chinese Language Teaching, 5, 47
  10. Kim, 2016, Effects of voice chat on EFL learners’ speaking ability according to proficiency levels, Multimedia Assisted Language Learning, 19, 63
  11. Koraishi, 2023, Teaching English in the age of AI: Embracing ChatGPT to optimize EFL materials and assessment, Language Education and Technology, 3, 55
  12. Li, H. (2018). The writing of academic introductions based on genre analysis and its teaching applications: A case study in the field of Chinese international education. [Master’s thesis, Zhejiang University].
  13. Li, 2020, A survey on the needs and curriculum development of academic Chinese writing courses for international undergraduate students: A case study of the general elective course at Peking University, International Chinese Education (Bilingual), 5, 51
  14. Li, 2023, Exploring the use of ChatGPT in Chinese language classrooms, International Journal of Chinese Language Teaching, 4, 36
  15. Liu, T. (2018). A literature review of master’s theses in Chinese international education for international students. [Master’s thesis, East China Normal University].
  16. Loncar, 2023, Multiple technologies, multiple sources: Trends and analyses of the literature on technology-mediated feedback for L2 English writing published from 2015–2019, Computer Assisted Language Learning, 36, 722, 10.1080/09588221.2021.1943452 · DOI
  17. Lu, 2006, Using instant messaging to provide an intelligent learning environment, Intelligent Tutoring Systems, Lecture Notes in Computer Science, 4053, 575
  18. Luan, 2023, Exploring the role of online EFL learners’ perceived social support in their learning engagement: A structural equation model, Interactive Learning Environments, 31, 1703, 10.1080/10494820.2020.1855211 · DOI
  19. Pica, 1991, Language learning through interaction: What role does gender play?, Studies in Second Language Acquisition, 13, 343, 10.1017/S0272263100010020 · DOI
  20. Shu, 2024, Effectiveness of Peer Feedback in Teaching Academic Chinese Writing for CSL Students, International Journal of Chinese Language Teaching, 6, 137
  21. Tadayyon, 2017, Exploring discourse markers used in academic papers: A comparative corpus-based inquiry of Iranian and English native writers, Iranian EFL Journal, 13, 130
  22. Tardy, 2010, Writing for the world: Wikipedia as an introduction to academic writing, English teaching forum, Vol. 48, 12
  23. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes, Harvard University Press.
  24. Zhang, 2022, Training in machine translation post-editing for foreign language students, Language Learning & Technology, 26, 1

Authors
Chen, C., & Gong, Y. (.
Affiliations
  • Southwest University (CN)
  • University of Macau (MO)
Corresponding author
Gong, Yang (Frank)
ORCID
Venue
Education Sciences
Publisher
MDPI AG
Published
2025-01-24
Volume
15
Issue
2
Pages
141
ISSN
2227-7102 (ISSN-L)
DOI
10.3390/educsci15020141
OpenAlex
W4406804344
Semantic Scholar
86acf6bddcd15742b98bf34d9d67ec1aa3acbb38
Cited by
90
References
56
Citations by year
2026: 58 · 2025: 32
FWCI
94.36
Type
journal article
Language
en
Open access
Licence
Locations
  • Education Sciences (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
University of Macau, Macau SAR, China (MYRG-GRG2024-00153-FED)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Physical Sciences › Computer Science › Information Systems › English Language Learning and Teaching
Keywords
AI-assisted learning, academic writing, Chinese as a second language, ChatGPT, international students, writing performance, student motivation, overreliance on AI, AI-generated content reliability, AI ethics in education, language teaching

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review