CoChat

Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students

Open access
Song, Cuiping, Song, Yanping · 2023 · Frontiers in Psychology

The DOI matches its Crossref record; 68 of the 71 references it cites were found in a registry or an index; 20 of the 47 sources its claims cite were checked for support.

Abstract

A mixed-methods study of 50 Chinese EFL students randomly assigned to AI-assisted ChatGPT versus traditional instruction, reporting significant improvements in writing skills and motivation.

Overview

CoChat Summary

TL;DR

This mixed-methods study found that 12 weeks of ChatGPT-assisted writing instruction improved Chinese EFL students’ academic writing skills and writing motivation more than traditional instruction. Qualitative interviews with experimental-group students were largely positive but raised concerns about contextual accuracy, over-reliance, and long-term sustainability.

Key findings

  • The experimental group outperformed the control group on global writing post-test after controlling for pre-test scores: M = 59.12 vs. 45.18; ANCOVA F(1, 47) = 19.01, p = 0.00, partial eta squared = 0.27; t-test t(48) = 3.79, p < 0.001, Cohen’s d = 0.76.
  • Specific writing dimensions favored the experimental group: content F(1, 47) = 12.61, p = 0.00, partial eta squared = 0.19; organization F(1, 47) = 23.16, p = 0.00, partial eta squared = 0.34; language use F(1, 47) = 9.86, p = 0.01, partial eta squared = 0.11.
  • Writing motivation was significantly higher in the experimental group: M = 20.06 vs. 18.21; ANCOVA F(1, 47) = 29.62, p = 0.00, partial eta squared = 0.38; t-test t(48) = 3.34, p = 0.001, Cohen’s d = 0.52.
  • Qualitative themes included positive perceptions of innovation and assistance, writing skill enhancements, motivation and engagement gains, recognized advantages such as accessibility and personalized feedback, acknowledged challenges including contextual accuracy and over-reliance, and concerns about long-term impact and sustainability.

Methods

  • Mixed-methods design with 50 Chinese EFL undergraduates at a national university, matched for proficiency and randomly assigned to experimental (ChatGPT-assisted instruction) or control (traditional teacher-led instruction) groups; intervention lasted 12 weeks, with AI sessions twice weekly for approximately 60 minutes.
  • Pre-test and post-test IELTS academic writing tasks 1 and 2 were scored with IELTS band descriptors by two raters (inter-rater reliability r = 0.88); writing motivation was measured with a 7-item Writing Motivation Scale on a 5-point Likert scale (Cronbach’s α = 0.86); semi-structured interviews were conducted with 9 experimental-group students for about 30 minutes each and analyzed thematically.
  • Quantitative analysis used one-way ANCOVA controlling for pre-test scores, independent samples t-tests, and effect sizes.

Limitations

  • Stated by the paper: generalizability is constrained by the specific sample of Chinese EFL students; the sample size of 50 is relatively small; the intervention duration was relatively short; possible contamination between experimental and control groups; no long-term follow-up to assess sustainability; and potential unintended plagiarism or over-reliance on AI-generated content despite ethical guidance.
  • Evident from the text: writing motivation was measured only by self-report questionnaire, and qualitative interviews were conducted only with experimental-group students, so control-group perceptions and experiences were not captured.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3389/fpsyg.2023.1260843

Cites

68 of 71 references verified0 citation errors · 3 not checkable

Statistics

10 tests recomputed · 0 with the reported and recomputed p on opposite sides of .05

Figures

Image 1
CC BY · From the PDF

Verification

Claims the paper makes

14 Valid · 3 Not valid · 17 Not checked · 81 Own results and interpretations

44 more claims

Create a free account to see all 115 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 26
  • Probably not a substantial claim. 42
  • Borderline: one of the tests for a substantial claim is unclear. 2
  • Not yet classified. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

13 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Statistics that don’t add up

Each reported p-value recomputed from its test statistic and degrees of freedom.

2 of 2

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Doesn’t add upt(48) = 3.01reported p = .003 · recomputed p = .004

    “t(48) = 3.01, p = .003”

  • Doesn’t add upt(48) = 3.34reported p = .001 · recomputed p = .002

    “t(48) = 3.34, p = .001”

  • Consistentt(48) = 3.79reported p < .001 · recomputed p < .001

    “t(48) = 3.79, p < .001”

  • Consistentt(48) = 4.12reported p < .001 · recomputed p < .001

    “t(48) = 4.12, p < .001”

  • Consistentt(48) = 4.34reported p < .001 · recomputed p < .001

    “t(48) = 4.34, p < .001”

5 of 10 · the rest aren’t loaded

5 more recomputed tests

Create a free account to see every test the Stats check recomputed.

See all tests — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

Show all 71 references

Research works printed but not linked: 2

  1. Pallant, 2013, SPSS survival manual
  2. Vygotsky, 1984, The problem of age, Pedagogical

Cited by

See all 717

Metadata

Authors
Song, C., & Song, Y.
Affiliations
  • North Minzu University (CN)
  • Central South University (CN)
Corresponding author
Song, Cuiping
Venue
Frontiers in Psychology
Published
2023-12-15
Volume
14
Pages
1260843-1260843
ISSN
1664-1078 (ISSN-L)
OpenAlex
W4389793309
Cited by
717
References
70
Citations by year
2026: 297 · 2025: 296 · 2024: 123 · 2023: 1
FWCI
802.96
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Psychology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Social Sciences › Education › Artificial Intelligence in Education
  • Social Sciences › Education › English Language Learning and Teaching
Keywords
ChatGPT, AI-assisted language learning, academic writing, writing skills, writing motivation, EFL learners, Chinese learners of English, generative AI in education, overreliance on AI, writing instruction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. https://app.cochat.ai/papers/10.3389/fpsyg.2023.1260843
The paper
Song, C., & Song, Y. (2023). Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2023.1260843

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.