CoChat

Evaluating the impact of using ChatGPT on EFL students’ argumentative skills

Open access
Mai, Hien Thi · 2026 · Discover Education

The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index; 11 of the 30 sources its claims cite were checked for support.

Abstract

It is important to look more closely at how generative AI tools like ChatGPT affect student writing as they become more common in the classroom. At a military boarding high school in Vietnam where students are not allowed to use personal electronic devices outside of class, this study examines the effects of ChatGPT on argumentative writing among intermediate-level EFL learners (CEFR B1–B2). A quasi-experimental design with little outside interference was made possible by this rigorous environment. 42 students, ages 16 to 17, were divided into two groups over the course of six weeks. While the control group received in-person teacher instruction, the experimental group supported writing tasks using ChatGPT in a supervised computer lab. Weekly writing essays were written by both groups, and their performance was evaluated using a rubric for pre- and post-tests. Semi-structured interviews, classroom observations, and reflection journals were used to collect qualitative data. The experimental group initially outperformed the control group in fluency and structure, according to the results; however, formulaic writing and a decreased level of critical engagement were indicators of an over-reliance on AI. In contrast, the control group gradually improved its originality, argumentation, and cognitive ownership despite starting out slower. Additionally, students in this group reported feeling more confident and growing personally. These results imply that, even though ChatGPT can improve writing at the surface level, it might impede deeper learning if instructional scaffolding isn’t used. For technology to complement critical thinking and learner autonomy rather than replace them, it must be carefully incorporated into pedagogical frameworks.

Overview

CoChat Summary

## TL;DR

This quasi-experimental mixed-methods study found that ChatGPT helped Vietnamese EFL students write more fluently and accurately, especially in lexical resource and grammatical range. However, teacher-led instruction produced stronger gains in task response, critical thinking, and authorial voice.

## Key findings

  • Both groups improved significantly from pre-test to post-test on all five rubric criteria. For the ChatGPT group (EG), gains were significant in Task Response, Coherence and Cohesion, Lexical Resource, Grammatical Range and Accuracy, and Critical Thinking, all with p < 0.001.
  • In post-test comparisons, EG scored significantly higher than the control group (CG) in Lexical Resource, t(40) = 7.95, p < 0.001, and Grammatical Range and Accuracy, t(40) = 10.86, p < 0.001.
  • CG scored significantly higher than EG in Task Response, t(40) = -3.13, p = 0.003, and Critical Thinking, t(40) = -6.16, p < 0.001; there was no significant difference in Coherence and Cohesion, t(40) = 0.99, p = 0.328.
  • The paper reports that EG’s early gains were mainly “surface-level” (e.g., grammar and coherence), while CG showed deeper development in idea development, originality, and personal voice over the six weeks.

## Methods

  • Quasi-experimental mixed-methods design with two intact groups of 11th-grade students in a Vietnamese military boarding high school over six weeks.
  • Sample: 42 students aged 16–17, split evenly into EG (n = 21; used ChatGPT under supervision) and CG (n = 21; received traditional teacher-led writing instruction). All were assessed as CEFR B1–B2.
  • Main analysis: pre/post argumentative essays scored by two trained raters on five criteria; the paper reports paired-samples and independent-samples t-tests. Qualitative data came from weekly reflection journals, semi-structured interviews with 10 students from each group, and classroom observations, analyzed thematically.

## Limitations

  • The paper states that, as with most quasi-experiments, external variables could not be fully controlled, which may limit generalizability.
  • The study was conducted in one military boarding high school in Vietnam with 42 students, so the narrow setting makes broader generalization difficult; this is evident from the text.
  • The intervention lasted six weeks, so the text suggests it captures short-term rather than long-term effects; this is evident from the text.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s44217-026-01153-1

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

10 Valid · 1 Not valid · 16 Not checked · 92 Own results and interpretations

32 more claims

Create a free account to see all 119 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 32
  • It may state two separate claims as one. 66
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentt(20) = -9.07reported p < .001 · recomputed p < .001

    “t(20) = -9.07, p < .001”

  • Consistentt(20) = -6.92reported p < .001 · recomputed p < .001

    “t(20) = -6.92, p < .001”

  • Consistentt(20) = -13.5reported p < .001 · recomputed p < .001

    “t(20) = -13.50, p < .001”

  • Consistentt(20) = -14.01reported p < .001 · recomputed p < .001

    “t(20) = -14.01, p < .001”

  • Consistentt(20) = -4.16reported p < .001 · recomputed p < .001

    “t(20) = -4.16, p < .001”

5 of 15 · the rest aren’t loaded

10 more recomputed tests

Create a free account to see every test the Stats check recomputed.

See all tests — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Citations

Cites

Show all 18 references

Research works printed but not linked: 5

  1. Creswell JW, Creswell JD. Research design: Qualitative, quantitative, and mixed methods approaches. 5th ed. Thousand Oaks, CA: SAGE; 2018.
  2. Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 3rd ed. Thousand Oaks, CA: SAGE; 2017.
  3. George AS. Preparing students for an AI-driven world: rethinking curriculum and pedagogy in the age of artificial intelligence. Partners Univers Innovative Res Publication. 2023;1(2):112–36.
  4. Hyland K. Second language writing. Cambridge: Cambridge University Press; 2019.
  5. Tahira M, Haider G. The role of critical thinking in academic writing: an investigation of EFL students’ perceptions and writing experiences. Int Online J Prim Educ. 2019;8(1):1–30.

Cited by

See all 2

Metadata

Authors
Mai, H. T.
Affiliations
Electric Power University (VN)
Corresponding author
Mai, Hien Thi
Venue
Discover Education
Published
2026-02-09
Volume
5
Issue
1
ISSN
2731-5525 (ISSN-L)
OpenAlex
W7128431959
Cited by
2
References
17
Citations by year
2026: 2
FWCI
39.88
Type
journal article
Language
en
Open access
Licence
Locations
  • Discover Education (publishedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Social Sciences › Education › Artificial Intelligence in Education
  • Social Sciences › Education › English Language Learning and Teaching
Keywords
ChatGPT, generative AI in education, EFL writing, argumentative writing, AI-assisted writing, overreliance on AI, formulaic writing, critical thinking, learner autonomy, instructional scaffolding, student self-confidence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Evaluating the impact of using ChatGPT on EFL students’ argumentative skills. https://app.cochat.ai/papers/10.1007/s44217-026-01153-1
The paper
Mai, H. T. (2026). Evaluating the impact of using ChatGPT on EFL students’ argumentative skills. Discover Education. https://doi.org/10.1007/s44217-026-01153-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.