Evaluating the impact of using ChatGPT on EFL students’ argumentative skills
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index; 11 of the 30 sources its claims cite were checked for support.
Abstract
It is important to look more closely at how generative AI tools like ChatGPT affect student writing as they become more common in the classroom. At a military boarding high school in Vietnam where students are not allowed to use personal electronic devices outside of class, this study examines the effects of ChatGPT on argumentative writing among intermediate-level EFL learners (CEFR B1–B2). A quasi-experimental design with little outside interference was made possible by this rigorous environment. 42 students, ages 16 to 17, were divided into two groups over the course of six weeks. While the control group received in-person teacher instruction, the experimental group supported writing tasks using ChatGPT in a supervised computer lab. Weekly writing essays were written by both groups, and their performance was evaluated using a rubric for pre- and post-tests. Semi-structured interviews, classroom observations, and reflection journals were used to collect qualitative data. The experimental group initially outperformed the control group in fluency and structure, according to the results; however, formulaic writing and a decreased level of critical engagement were indicators of an over-reliance on AI. In contrast, the control group gradually improved its originality, argumentation, and cognitive ownership despite starting out slower. Additionally, students in this group reported feeling more confident and growing personally. These results imply that, even though ChatGPT can improve writing at the surface level, it might impede deeper learning if instructional scaffolding isn’t used. For technology to complement critical thinking and learner autonomy rather than replace them, it must be carefully incorporated into pedagogical frameworks.
Overview
CoChat Summary
## TL;DR
This quasi-experimental mixed-methods study found that ChatGPT helped Vietnamese EFL students write more fluently and accurately, especially in lexical resource and grammatical range. However, teacher-led instruction produced stronger gains in task response, critical thinking, and authorial voice.
## Key findings
- Both groups improved significantly from pre-test to post-test on all five rubric criteria. For the ChatGPT group (EG), gains were significant in Task Response, Coherence and Cohesion, Lexical Resource, Grammatical Range and Accuracy, and Critical Thinking, all with p < 0.001.
- In post-test comparisons, EG scored significantly higher than the control group (CG) in Lexical Resource, t(40) = 7.95, p < 0.001, and Grammatical Range and Accuracy, t(40) = 10.86, p < 0.001.
- CG scored significantly higher than EG in Task Response, t(40) = -3.13, p = 0.003, and Critical Thinking, t(40) = -6.16, p < 0.001; there was no significant difference in Coherence and Cohesion, t(40) = 0.99, p = 0.328.
- The paper reports that EG’s early gains were mainly “surface-level” (e.g., grammar and coherence), while CG showed deeper development in idea development, originality, and personal voice over the six weeks.
## Methods
- Quasi-experimental mixed-methods design with two intact groups of 11th-grade students in a Vietnamese military boarding high school over six weeks.
- Sample: 42 students aged 16–17, split evenly into EG (n = 21; used ChatGPT under supervision) and CG (n = 21; received traditional teacher-led writing instruction). All were assessed as CEFR B1–B2.
- Main analysis: pre/post argumentative essays scored by two trained raters on five criteria; the paper reports paired-samples and independent-samples t-tests. Qualitative data came from weekly reflection journals, semi-structured interviews with 10 students from each group, and classroom observations, analyzed thematically.
## Limitations
- The paper states that, as with most quasi-experiments, external variables could not be fully controlled, which may limit generalizability.
- The study was conducted in one military boarding high school in Vietnam with 42 students, so the narrow setting makes broader generalization difficult; this is evident from the text.
- The intervention lasted six weeks, so the text suggests it captures short-term rather than long-term effects; this is evident from the text.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
10 Valid · 1 Not valid · 16 Not checked · 92 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Recent studies on online peer feedback and AI-supported collaborative writing have demonstrated that structured interaction plays a critical role in fostering argumentative quality and learner engagement [2].”
§ 2 Literature review > 2.2 AI-assisted writing: opportunities and concerns
What the sources say
- Partly supportedcites [2] Banihashem SK, Kerman NT, Noroozi O, et al. Feedback sources in essay writing: peer-generated or AI-generated feedback? Int J Educ Technol High Educ. 2024;21:23. https://doi.org/10.1186/s41239-024-00455-4. sourceExperimental — accuracy not yet measured
The Introduction says peer feedback can increase students’ active engagement, and the Study design and procedure describes a structured prompt for feedback on argumentative essays. But the Abstract reports a comparison of peer- and ChatGPT-generated feedback quality, not a demonstration that structured interaction improves argumentative writing quality or engagement in AI-supported collaborative writing. Page numbers are unavailable in our copy. (LLM, conf=0.87)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Few studies have looked at the long-term effects of ChatGPT and similar tools on the development of writing skills, particularly in actual classroom settings.”
§ 2 Literature review > 2.5 Gaps in current research
What the sources say
- Partly supportedcites Chen et al. [5] Chen S, Shi P, Guo J, Xu S, Tian J. From process to product: writing engagement and performance of EFL learners under computer-generated feedback instruction. Front Psychol. 2023;14:1258286. https://doi.org/10.3389/fpsyg.2023.1258286. sourceExperimental — accuracy not yet measured
The Abstract says that the impact of automated writing feedback on students’ writing quality remains under-explored. But the Introduction describes a wealth of research on automated writing evaluation, including studies of short- and long-term writing quality, and the literature review identifies a classroom study. The paper does not establish that few studies have examined the long-term effects of ChatGPT or similar tools in classrooms. (LLM, conf=0.72)
- Partly supportedcites Tran [22] Tran TTT. Enhancing EFL writing revision practices: the impact of AI- and teacher-generated feedback and their sequences. Educ Sci. 2025;15(2):232. https://doi.org/10.3390/educsci15020232. sourceExperimental — accuracy not yet measured
The Conclusions (section 6; page unavailable) call for further research on the long-term effects of integrated AI- and teacher-generated feedback, supporting the existence of a research gap. The paper does not say that few studies examine the long-term effects of ChatGPT and similar tools on writing-skill development, particularly in actual classrooms; its stated gap is narrower. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This study found that ChatGPT and other AI tools can help students produce more grammatically correct and fluent texts, which is in line with previous research [9, 17].”
What the sources say
- Partly supportedcites [9] Dizon G, Gold J. Exploring the effects of grammarly on EFL students’ foreign Language anxiety and learner autonomy. JALT CALL J. 2023;19(3):299–316. sourceExperimental — accuracy not yet measured
The Automated writing evaluation section (page unavailable) reports improved grammatical accuracy with Grammarly, an AI-based writing tool. It does not address ChatGPT, and the study it summarizes found no statistically significant difference in writing fluency between Grammarly and non-Grammarly groups. Thus the source supports the grammatical-correctness part, but not the ChatGPT or fluency parts. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [17] McGrath C, Pargman TC, Juth N, Palmgren PJ. University teachers’ perceptions of responsibility and artificial intelligence in higher education-An experimental philosophical study. Computers Education: Artif Intell. 2023;4:100139. source
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The quantitative analysis showed that while both groups made progress in argumentative writing performance”
§ 4 Results > 4.1 Quantitative results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both EG and CG demonstrated significant gains from pre-test to post-test across all five evaluated criteria.”
§ 4 Results > 4.1 Quantitative results
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
32 more claims
Create a free account to see all 119 claims and every source verdict.
- Probably not a substantial claim. 32
- It may state two separate claims as one. 66
- The quote only partly states this claim. 2
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentt(20) = -9.07reported p < .001 · recomputed p < .001
“t(20) = -9.07, p < .001”
- Consistentt(20) = -6.92reported p < .001 · recomputed p < .001
“t(20) = -6.92, p < .001”
- Consistentt(20) = -13.5reported p < .001 · recomputed p < .001
“t(20) = -13.50, p < .001”
- Consistentt(20) = -14.01reported p < .001 · recomputed p < .001
“t(20) = -14.01, p < .001”
- Consistentt(20) = -4.16reported p < .001 · recomputed p < .001
“t(20) = -4.16, p < .001”
10 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
A process genre approach to teaching writing
Badger, R., & White, G. (2000) · ELT Journal · doi:10.1093/elt/54.2.153 · Cited by 756
Critical English for Academic Purposes: Theory, Politics, and Practice
Benesch, S. (2001) · Routledge eBooks · doi:10.4324/9781410601803 · Cited by 581
Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M., & Chizari, M. (2012) · Educational Research Review · doi:10.1016/j.edurev.2011.11.006 · Cited by 310
Feedback sources in essay writing: peer-generated or AI-generated feedback?
Banihashem, S. K., Kerman, N. T., Noroozi, O., Moon, J., & Drachsler, H. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00455-4 · Cited by 282
Moorhouse, B. L., Wan, Y., Wu, C., Kohnke, L., Ho, T. Y., & Kwong, T. (2024) · System · doi:10.1016/j.system.2024.103399 · Cited by 228
McGrath, C., Cerratto Pargman, T., Juth, N., & Palmgren, P. J. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100139 · Cited by 195
Dimensions of difference: a comparison of university writing and IELTS writing
Moore, T., & Morton, J. (2005) · Journal of English for Academic Purposes · doi:10.1016/j.jeap.2004.02.001 · Cited by 135
Yin, X. F., Saad, M. R. B. M., & Halim, H. B. A. (2023) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2023.101363 · Cited by 92
Research works printed but not linked: 5
- Creswell JW, Creswell JD. Research design: Qualitative, quantitative, and mixed methods approaches. 5th ed. Thousand Oaks, CA: SAGE; 2018.
- Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 3rd ed. Thousand Oaks, CA: SAGE; 2017.
- George AS. Preparing students for an AI-driven world: rethinking curriculum and pedagogy in the age of artificial intelligence. Partners Univers Innovative Res Publication. 2023;1(2):112–36.
- Hyland K. Second language writing. Cambridge: Cambridge University Press; 2019.
- Tahira M, Haider G. The role of critical thinking in academic writing: an investigation of EFL students’ perceptions and writing experiences. Int Online J Prim Educ. 2019;8(1):1–30.
Cited by
Apeadido, S., Owusu-Fordjour, C., & Mahama, A. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02113-2
“A core methodological protocol was strictly enforced: students were prohibited from direct copy-pasting, requiring them to summarise all AI-generated guidance in their physical notebooks [39, 79].”
Metadata
- Authors
- Mai, H. T.
- Affiliations
- Electric Power University (VN)
- Corresponding author
- Mai, Hien Thi
- ORCID
- Mai, Hien Thi
- Venue
- Discover Education
- Published
- 2026-02-09
- Volume
- 5
- Issue
- 1
- ISSN
- 2731-5525 (ISSN-L)
- OpenAlex
- W7128431959
- Semantic Scholar
- 40bb0a553c535f51b6a6add9b124f97ff54b451e
- Cited by
- 2
- References
- 17
- Citations by year
- 2026: 2
- FWCI
- 39.88
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor, 32 d)
- Locations
- Discover Education (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Education › English Language Learning and Teaching
- Keywords
- ChatGPT, generative AI in education, EFL writing, argumentative writing, AI-assisted writing, overreliance on AI, formulaic writing, critical thinking, learner autonomy, instructional scaffolding, student self-confidence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluating the impact of using ChatGPT on EFL students’ argumentative skills. https://app.cochat.ai/papers/10.1007/s44217-026-01153-1
- The paper
- Mai, H. T. (2026). Evaluating the impact of using ChatGPT on EFL students’ argumentative skills. Discover Education. https://doi.org/10.1007/s44217-026-01153-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.