The influence of GenAI on the effectiveness of argumentative writing in higher education: evidence from a quasi-experimental study in China
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
The revolutionary development of generative artificial intelligence, especially large language model (LLM) chatbots like ChatGPT, has brought disruptive changes to society, especially in the field of higher education, particularly in the domain of writing. Prior studies have primarily examined the opportunities and challenges that LLM Chatbots bring to higher education writing from a theoretical perspective. However, empirical studies on how to effectively and responsibly integrate LLM Chatbots into writing learning and evaluate their effectiveness are still limited. To address this gap, the present study introduces a learning approach termed LLM-powered Chatbot-assisted argumentative writing (LCAW) to support the cultivation of university students’ writing abilities. A total of 61 Chinese university students from two classes were invited to participate in a quasi-experimental study with different learning methods as intervention measures, and student perceptions were gathered through interviews. The results show that the proposed LCAW method significantly improves students’ writing performance in terms of content quality and language expression, and positively impacts in providing personalized feedback and enhancing writing motivation. This study contributes by proposing an effective method for integrating generative AI into higher education writing instruction and by offering insights for curriculum design policies to enhance higher writing education through GenAI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 2×
“…data, these tools had enhanced abilities in natural language understanding, processing long inputs, providing creative responses to various prompts, and maintaining natural dialogue flows through repeated training and refinement (Adamopoulou & Moussiades, 2020; Floridi, 2023; Kasneci et al., 2023).”
Kuhn, D. (1991) · doi:10.1017/cbo9780511571350 · Cited by 2,094
Vision, challenges, roles and research issues of Artificial Intelligence in Education
Hwang, G.-J., Xie, H., Wah, B. W., & Gašević, D. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100001 · Cited by 1,435 · cited 1×
“As different functions of artificial intelligence technology often imply various pedagogical perspectives, researchers need to construct targeted frameworks or models based on learning theories to achieve diverse teaching objectives (Hwang et al., 2020).”
A SWOT analysis of ChatGPT: Implications for educational practice and research
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2195846 · Cited by 1,384 · cited 1×
“…have highlighted the significant potential of large language models in various writing tasks, such as facilitating idea generation (Atlas, 2023), providing mentorship to enhance students’ writing skills (Farrokhnia et al., 2023), and assisting in editing and assessment (Mizumoto & Eguchi, 2023).”
Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023) · The International Journal of Management Education · doi:10.1016/j.ijme.2023.100790 · Cited by 1,261
Chatbots: History, technology, and applications
Adamopoulou, E., & Moussiades, L. (2020) · Machine Learning with Applications · doi:10.1016/j.mlwa.2020.100006 · Cited by 1,090 · cited 1×
“…data, these tools had enhanced abilities in natural language understanding, processing long inputs, providing creative responses to various prompts, and maintaining natural dialogue flows through repeated training and refinement (Adamopoulou & Moussiades, 2020; Floridi, 2023; Kasneci et al., 2023).”
ChatGPT User Experience: Implications for Education
Zhai, X. (2023) · doi:10.2139/ssrn.4312418 · Cited by 935 · cited 1×
“Conversely, certain studies have raised concerns regarding the integration of large language models in higher education writing, citing issues related to tool reliability (Harrer, 2023), academic integrity (Zhai, 2022), and potential over-reliance on the technology (Kasneci et al., 2023).”
Using ChatGPT for second language writing: pitfalls and potentials
Barrot, J. S. (2023) · Assessing Writing · doi:10.1016/j.asw.2023.100745 · Cited by 770 · cited 2×
“This finding aligns with the results of Yan (2023) and Barrot (2023), suggesting that large language models may produce inaccurate and irrelevant responses.”
Cited by
Liu, X., Guo, B., He, W., & Hu, X. (2025) · Journal of Educational Computing Research · doi:10.1177/07356331251329185 · Cited by 89
University policies on generative AI in Asia: promising practices, gaps, and future directions
Dai, Y., Lai, S., Lim, C. P., & Liu, A. (2025) · Journal of Asian Public Policy · doi:10.1080/17516234.2024.2379070 · Cited by 68
Xia, Q., Li, W., Yang, Y., Weng, X., & Chiu, T. K. F. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100455 · Cited by 45
Zhang, M., & Crosthwaite, P. (2026) · International Review of Applied Linguistics in Language Teaching · doi:10.1515/iral-2024-0196 · Cited by 26
“For instance, Li et al. (2024) analysed the vocabulary usage of GPT in academic English witing and discovered that it excels in terms of lexical diversity and precision.”
Bo, Z., Pek, L. S., Jian, L., Cong, W., & Nan, L. T. (2025) · Language Teaching Research Quarterly · doi:10.32038/ltrq.2025.53.05 · Cited by 6
Metadata
- Authors
- Li, H., Wang, Y., Luo, S., & Huang, C.
- Venue
- Journal of Asian Public Policy
- Publisher
- Informa UK Limited
- Published
- 2024-06-06
- Volume
- 18
- Issue
- 2
- Pages
- 405-430
- ISSN
- 1751-6234 (ISSN-L)
- 1751-6242
- OpenAlex
- W4399441897
- Cited by
- 60
- References
- 36
- FWCI
- 22.96
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The influence of GenAI on the effectiveness of argumentative writing in higher education: evidence from a quasi-experimental study in China. https://app.cochat.ai/papers/10.1080/17516234.2024.2363128
- The paper
- Li, H., Wang, Y., Luo, S., & Huang, C. (2025). The influence of GenAI on the effectiveness of argumentative writing in higher education: evidence from a quasi-experimental study in China. Journal of Asian Public Policy. https://doi.org/10.1080/17516234.2024.2363128
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.