The Impact of ChatGPT on Academic Writing Skills and Knowledge: An Investigation of Its Use in Argumentative Essays
Open accessThe DOI matches its Crossref record; 43 of the 45 references it cites were found in a registry or an index, and 2 are not checked yet.
Abstract
This study investigates the impact of ChatGPT on EFL students' argumentative writing development of 30 third-year English majors. The research examines writing quality improvements, student perceptions, and patterns of AI tool usage across a 16-week period. This study employed a mixed-methods design to investigate both students' writing development and their perceptions of AI integration. Writing development was assessed through comparative analysis of participants' first and fifth drafts as pre-test and post-test assessments, while perceptions data were collected through a validated 17-item questionnaire (Cronbach's α = 0.89) focusing on confidence and concerns in AI-assisted writing. Essays were evaluated using an adapted AIAS framework focusing on six dimensions: content, organization, language use, critical thinking, AI integration, and academic integrity. The quantitative analysis revealed significant enhancements in writing performance, with the most substantial improvements in academic integrity (+3.0 points. Furthermore, the thematic analysis of the draft comparisons demonstrated marked progression in argument construction, evidence integration, and academic voice. This research provides insights for developing pedagogical frameworks that optimize AI integration while fostering independent writing competencies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“For example, studies examining AI-assisted writing tasks report that students often produce more fluent and well-structured text, while improvements in argument quality or conceptual reasoning remain limited [10][11][12].”
From Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings (2026) · cites this paper as [12]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Intervention studies have shown that using ChatGPT as a formative feedback tool can enhance writing accuracy, organization, and learner engagement while reducing anxiety (Khampusaen, 2025; Mahapatra, 2024).”
From Perceptions vs. practices: academic integrity and actual ChatGPT use among EFL students (2026) · cites this paper as Khampusaen, 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“align with the intervention results of (Khampusaen 2025) and (Mahapatra 2024), who reported that ethically integrated AI feedback enhanced writing fluency and confidence.”
From Perceptions vs. practices: academic integrity and actual ChatGPT use among EFL students (2026) · cites this paper as Khampusaen 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Full authorship substitution AI generates most or all of the final text High-quality surface output with minimal effort Limited development of writing, reasoning, and conceptual understanding [12,37]”
From Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings (2026) · cites this paper as [12]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reflecting on reflexive thematic analysis
Braun, V., & Clarke, V. (2019) · Qualitative Research in Sport, Exercise and Health · doi:10.1080/2159676x.2019.1628806 · Cited by 19,677 · cited 1×
“This began when the essays were imported into NVivo 14, where a thematic analysis was conducted through a systematic coding process following the Braun and Clarke' (2019) six-phase framework.”
Research design: qualitative, quantitative, and mixed methods approaches
Caldas, M. P. (2003) · Revista de Administração Contemporânea · doi:10.1590/s1415-65552003000100015 · Cited by 18,745 · cited 2×
“Second, their successful completion of fundamental writing courses ensured a basic understanding of academic writing conventions, providing a solid foundation for the study (Creswell & Creswell, 2018).”
Cohen, L., Manion, L., & Morrison, K. (2017) · doi:10.4324/9781315456539 · Cited by 6,018 · cited 1×
“The pre-post intervention design, as outlined by Cohen et al (2017), provides a foundational methodology for measuring writing development.”
Toward a Psychology of Human Agency
Bandura, A. (2006) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2006.00011.x · Cited by 3,459
Chatting about ChatGPT: how may AI and GPT impact academia and libraries?
Lund, B. D., & Wang, T. (2023) · Library Hi Tech News · doi:10.1108/lhtn-01-2023-0009 · Cited by 1,355
Discovering Statistics Using IBM SPSS Statistics: And Sex and Drugs and Rock “N” Roll
(2014) · Pflege · doi:10.1024/1012-5302/a000397 · Cited by 1,224 · cited 1×
“ANOVA is applied using IBM SPSS Statistics 29 to examine differences in argument quality across various groups, following Field's (2009) statistical procedures.”
Chatting about ChatGPT: How May AI and GPT Impact Academia and Libraries?Preprint
Lund, B. D., & Wang, T. (2023) · Social Science Research Network · doi:10.2139/ssrn.4333415 · Cited by 1,093 · cited 2×
“It is however necessary to consider ongoing evaluation and adjustment to ensure they effectively serve both pedagogical objectives and student needs, as emphasized by Lund et al. (2023) in their analysis of AI integration challenges and opportunities in academic settings.”
Hyland, K. (2003) · Cambridge University Press eBooks · doi:10.1017/cbo9780511667251 · Cited by 1,067
Perkins, M. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.02.07 · Cited by 839
Teaching by Principles: An Interactive Approach to Language Pedagogy
Lander, J. A., & Brown, H. D. (1995) · Language · doi:10.2307/415773 · Cited by 826 · cited 1×
“First, as third-year English majors, these students demonstrated the required language proficiency level necessary for engaging in complex argumentative writing tasks (Brown & Lee, 2015).”
Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation
Yan, D. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11742-4 · Cited by 713
Collaborating with ChatGPT in argumentative writing classrooms
Su, Y., Lin, Y., & Lai, C. (2023) · Assessing Writing · doi:10.1016/j.asw.2023.100752 · Cited by 563
Impact of AI assistance on student agency
Darvishi, A., Khosravi, H., Sadiq, S., Gašević, D., & Siemens, G. (2024) · Computers & Education · doi:10.1016/j.compedu.2023.104967 · Cited by 548 · cited 1×
“…contemporary scholars have also highlighted the pedagogical challenges in establishing an optimal equilibrium between AI-enhanced scaffolding and autonomous learning, particularly in EFL contexts, where metacognitive development is crucial (Darvishi et al., 2024; Jia & Tu, 2024; Nguyen, 2024).”
Wei, L. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1261955 · Cited by 536
Marzuki, Widiati, U., Rusdin, D., Darwin, & Indrawati, I. (2023) · Cogent Education · doi:10.1080/2331186x.2023.2236469 · Cited by 524 · cited 3×
“To sum up, while ChatGPT has successfully demonstrated promising potential for enhancing writing skills (Marzuki et al., 2023), there remains insufficient research examining how systematic AI integration affects the development of argumentative writing skills over time, particularly in EFL contexts…”
Ferris, D. R. (2010) · Studies in Second Language Acquisition · doi:10.1017/s0272263109990490 · Cited by 523
Comparing the quality of human and ChatGPT feedback of students’ writing
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.101894 · Cited by 509
The Psychology of Written Composition
(2013) · doi:10.4324/9780203812310 · Cited by 505
Questionnaires in Second Language Research: Construction, Administration, and Processing
Dörnyei, Z., & Taguchi, T. (2009) · Routledge eBooks · doi:10.4324/9780203864739 · Cited by 470
Flanagin, A., Bibbins-Domingo, K., Berkwits, M., & Christiansen, S. L. (2023) · JAMA · doi:10.1001/jama.2023.1344 · Cited by 420 · cited 1×
“Furthermore, studies examining the sustained effects of AI assistance on independent writing capabilities are notably scarce (Flanagin et al., 2023; Liu & Fan, 2024; Nguyen et al., 2024).”
Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study
Mahapatra, S. (2024) · Smart Learning Environments · doi:10.1186/s40561-024-00295-9 · Cited by 398 · cited 2×
“For example, research by Livberber and Ayvaz (2023), and Mahapatra (2024) highlights the specific improvements in grammar, mechanics, and organizational skills obtained through AI assistance.”
Perkins, M., Furze, L., Roe, J., & MacVaugh, J. (2024) · Journal of University Teaching and Learning Practice · doi:10.53761/q3azde36 · Cited by 291
AI in the Foreign Language Classroom: A Pedagogical Overview of Automated Writing Assistance Tools
Alharbi, W. (2023) · Education Research International · doi:10.1155/2023/4253331 · Cited by 286
Arguing to learn and learning to argue: design justifications and guidelines
Jonassen, D. H., & Kim, B. (2010) · Educational Technology Research and Development · doi:10.1007/s11423-009-9143-8 · Cited by 283 · cited 1×
“For example, Marzuki et al. (2023) documented significant gains in writing fluency (Marzuki et al., 2023), while Jonassen and Kim (2010) demonstrated improvements in argumentative structure.”
Feedback sources in essay writing: peer-generated or AI-generated feedback?
Banihashem, S. K., Kerman, N. T., Noroozi, O., Moon, J., & Drachsler, H. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00455-4 · Cited by 282
Liu, M., Zhang, L. J., & Biebricher, C. (2024) · Computers & Education · doi:10.1016/j.compedu.2023.104977 · Cited by 282
A systematic review of AI-based automated written feedback research
Shi, H., & Aryadoust, V. (2024) · ReCALL · doi:10.1017/s0958344023000265 · Cited by 222
Jia, X.-H., & Tu, J.-C. (2024) · Systems · doi:10.3390/systems12030074 · Cited by 210 · cited 1×
“…contemporary scholars have also highlighted the pedagogical challenges in establishing an optimal equilibrium between AI-enhanced scaffolding and autonomous learning, particularly in EFL contexts, where metacognitive development is crucial (Darvishi et al., 2024; Jia & Tu, 2024; Nguyen, 2024).”
Using chatbots to scaffold EFL students’ argumentative writing
Guo, K., Wang, J., & Chu, S. K. W. (2022) · Assessing Writing · doi:10.1016/j.asw.2022.100666 · Cited by 201 · cited 1×
“The focus on argumentative writing is deliberate and significant because this type of academic writing demands advanced cognitive and linguistic skills, crucial for success in higher education and professional environments (Chala & Chapetón, 2012; Guo et al., 2022).”
Horta, H., & Santos, J. M. (2016) · Research in Higher Education · doi:10.1007/s11162-015-9380-0 · Cited by 192 · cited 1×
“…systematic investigation into the cognitive mechanisms through which AI tools influence the argument construction processes and their subsequent impact on learners' critical thinking development (Alasgarova & Rzayev, 2024; Horta & Santos, 2015; Mahmood Khan, 2024; Slimi et al., 2022; Wei, 2023).”
Liu, H., & Fan, J. (2024) · European Journal of Education · doi:10.1111/ejed.12813 · Cited by 178
Eltahir, M. E., & Babiker, F. M. E. (2024) · The Electronic Journal of e-Learning · doi:10.34190/ejel.22.9.3639 · Cited by 112
The impact of Artificial Intelligence in academia: Views of Turkish academics on ChatGPT
Livberber, T., & Ayvaz, S. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e19688 · Cited by 103 · cited 4×
“Similar patterns of improvement in EFL writing development through AI assistance were also documented by Livberber and Ayvaz (2023).”
Influential20 years of technology and language assessment in Language Learning & Technology
Chapelle, C., & Voss, E. (2016) · Language Learning & Technology · doi:10.64152/10125/44464 · Cited by 101
Dashti, M., Londono, J., Ghasemi, S., & Moghaddasi, N. (2023) · Journal of Prosthetic Dentistry · doi:10.1016/j.prosdent.2023.05.023 · Cited by 66
Kim, Y. G., Park, C., & Park, Y. (2015) · Reading and Writing · doi:10.1007/s11145-015-9542-7 · Cited by 65
Nguyen, H. A. (2024) · International journal of AI in language education. · doi:10.54855/ijaile.24122 · Cited by 54
Problematizing Argumentative Writing in an Iranian EFL Undergraduate Context
Ghanbari, N., & Salari, M. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.862400 · Cited by 44 · cited 1×
“According to Ghanbari & Salari (2022), EFL students often find it difficult to create clear arguments, address opposing viewpoints, and provide evidence for their points while also dealing with language barriers.”
ChatGPT and Beyond: The Generative AI Revolution in EducationPreprint
Alsmadi, M. (2023) · arXiv.org · doi:10.48550/arxiv.2311.15198 · Cited by 38 · cited 3×
“…writing instruction methods have shown limited effectiveness in addressing these challenges (Patty, 2024), ChatGPT, as an innovative possibility for transforming writing pedagogy, requires a systematic evaluation of its impact on EFL learners' argumentative writing development (AL-Smadi, 2023) .”
InfluentialThe Role of Artificial Intelligence in Shaping High School Students' Motivation
Alasgarova, R., & Rzayev, J. (2024) · International Journal of Technology in Education and Science · doi:10.46328/ijtes.553 · Cited by 33
Toward non-human-centered design: designing an academic article with ChatGPT
LİVBERBER, T. (2023) · El Profesional de la Informacion · doi:10.3145/epi.2023.sep.12 · Cited by 30 · cited 1×
“…demonstrated promising potential for enhancing writing skills (Marzuki et al., 2023), there remains insufficient research examining how systematic AI integration affects the development of argumentative writing skills over time, particularly in EFL contexts (Dashti et al., 2023; Livberber, 2023).”
"EFL argumentative essay writing as a situated-social practice: A review of concepts"
Javeriana, P. U., Bejarano, P. A. C., Chapetón, C. M., & Nacional, U. P. (2012) · Folios · doi:10.17227/01234870.36folios23.36 · Cited by 27 · cited 1×
“The focus on argumentative writing is deliberate and significant because this type of academic writing demands advanced cognitive and linguistic skills, crucial for success in higher education and professional environments (Chala & Chapetón, 2012; Guo et al., 2022).”
Srinawati, W., & Alwi, R. (2020) · JURNAL SERAMBI ILMU · doi:10.32672/si.v21i2.2194 · Cited by 17 · cited 1×
“For EFL learners, argumentative writing presents unique challenges due to its requirement for simultaneous mastery of advanced language skills along with critical thinking abilities (Srinawati & Alwi, 2020).”
Writing Issues in ESL and their Potential Solutions: Case Study IMCO's Foundation Students
Slimi, Z., Alawai, F. A., Alyani, H. A., Abri, S. A., Al-Farsi, F. A., & Balushi, K. A. (2022) · Journal of Educational and Social Research · doi:10.36941/jesr-2022-0146 · Cited by 10 · cited 1×
“…systematic investigation into the cognitive mechanisms through which AI tools influence the argument construction processes and their subsequent impact on learners' critical thinking development (Alasgarova & Rzayev, 2024; Horta & Santos, 2015; Mahmood Khan, 2024; Slimi et al., 2022; Wei, 2023).”
Liu, T., & Taresh, S. M. (2024) · Journal of Ecohumanism · doi:10.62754/joe.v3i7.4520 · Cited by 7
Research works printed but not linked: 8
- Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. SAGE Publications.
- Etikan, I. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5, 1. https://doi.org/10.11648/j.ajtas.20160501. · doi:10.11648/j.ajtas.20160501
- Mahmood Khan, W. (2024). Analyzing the AI tools' impact on critical thinking in BS English students at Pakistani Universities. Journal of Applied Linguistics and TESOL, 7(4).
- Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook, 2nd ed. (pp. xiv, 338). Sage Publications, Inc.
- Nguyen, T., Lai, N., & Nguyen, Q. (2024). Artificial intelligence (AI) in education: A case study on ChatGPT's influence on student learning behaviors. Educational Process International Journal, 13, 105-121. https://doi.org/10.22521/edupij.2024.132. · doi:10.22521/edupij.2024.132
- Patty, J. (2024). Addressing student writing challenges: A review of difficulties and effective strategies. Education Journal: Journal Educational Research and Development, 8, 369-392. https://doi.org/10.31537/ej.v8i2.1938 · doi:10.31537/ej.v8i2.1938
- Saldana, J. (2013). The coding manual for qualitative researchers (2nd ed.). SAGE Publications Ltd. https://emotrab.ufba.br/wp- content/uploads/2020/09/Saldana-2013- TheCodingManualforQualitativeResearchers.pdf
- Zou, B., & Lertlit, S. (2022). Oxford's strategy inventory for language learning: English learning of Chinese students in Thai University. Language Education and Acquisition Research Network, 15(2), 705-723.
Cited by
Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings
Kumar, S., Mikayelyan, A., & Vorfolomeyeva, O. (2026) · Information · doi:10.3390/info17030299 · Cited by 11
Perceptions vs. practices: academic integrity and actual ChatGPT use among EFL students
Alsadoon, R. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1796737 · Cited by 8
“Intervention studies have shown that using ChatGPT as a formative feedback tool can enhance writing accuracy, organization, and learner engagement while reducing anxiety (Khampusaen, 2025; Mahapatra, 2024).”
Li, X., & Awang, M. M. B. (2026) · European Journal of Education · doi:10.1111/ejed.70490 · Cited by 7
Zhang, R., Zou, D., Xie, H., & Wang, F. L. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100489 · Cited by 9
Anthony, A., Sharma, S., & Sharma, S. (2025) · Bulletin of Science Technology & Society · doi:10.1177/02704676251353105 · Cited by 12
“ChatGPT has emerged as a powerful academic aid, streamlining tasks like brainstorming, drafting, and editing (Burger et al., 2023; Khampusaen, 2025; Siegerink et al., 2023).”
Metadata
- Authors
- Khampusaen, D.
- Affiliations
- Khon Kaen University (TH)
- Corresponding author
- Khampusaen, Dararat
- ORCID
- Khampusaen, Dararat
- Venue
- LEARN Journal: Language Education and Acquisition Research Network
- Publisher
- Thammasat University
- Published
- 2025-01-31
- Volume
- 18
- Issue
- 1
- Pages
- 963-988
- ISSN
- 2630-0672 (ISSN-L)
- 2672-9431
- OpenAlex
- W4407061908
- Cited by
- 31
- References
- 30
- Citations by year
- 2026: 24 · 2025: 7
- FWCI
- 32.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- LEARN Journal : Language Education and Acquisition Research Network (publishedVersion, cc-by)
- Funders
- Khon Kaen University
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Education › English Language Learning and Teaching
- Keywords
- ChatGPT, AI-assisted writing, argumentative essay writing, EFL writing, academic writing skills, academic integrity, student perceptions of AI, writing development, evidence integration, academic voice, critical thinking, English majors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
