Cited by 27Artificial Intelligence in EducationWriting and Handwriting EducationEFL/ESL Teaching and Learning

Modifying AI, Enhancing Essays: How Active Engagement with Generative AI Boosts Writing Quality

Open access
Yang, Kaixun, Raković, Mladen, Liang, Zhiping, Yan, Lixiang, Zeng, Zijie, Fan, Yizhou, Gašević, Dragan, Chen, Guanliang·2025·Proceedings of the 15th International Learning Analytics and Knowledge Conference

Abstract

Students are increasingly relying on Generative AI (GAI) to support their writing-a key pedagogical practice in education. In GAI-assisted writing, students can delegate core cognitive tasks (e.g., generating ideas and turning them into sentences) to GAI while still producing high-quality essays. This creates new challenges for teachers in assessing and supporting student learning, as they often lack insight into whether students are engaging in meaningful cognitive processes during writing or how much of the essay's quality can be attributed to those processes. This study aimed to help teachers better assess and support student learning in GAI-assisted writing by examining how different writing behaviors, especially those indicative of meaningful learning versus those that are not, impact essay quality. Using a dataset of 1,445 GAI-assisted writing sessions, we applied the cutting-edge method, X-Learner, to quantify the causal impact of three GAI-assisted writing behavioral patterns (i.e., seeking suggestions but not accepting them, seeking suggestions and accepting them as they are, and seeking suggestions and accepting them with modification) on four measures of essay quality (i.e., lexical sophistication, syntactic complexity, text cohesion, and linguistic bias). Our analysis showed that writers who frequently modified GAI-generated text-suggesting active engagement in higher-order cognitive processes-consistently improved the quality of their essays in terms of lexical sophistication, syntactic complexity, and text cohesion. In contrast, those who often accepted GAI-generated text without changes, primarily engaging in lower-order processes, saw a decrease in essay quality. Additionally, while human writers tend to introduce linguistic bias when writing independently, incorporating GAI-generated text-even without modification-can help mitigate this bias.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3706468.3706544

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The quality of GenAI output has been shown to be enhanced through an iterative interaction with the user.

    “In terms of professional output, GenAI has been found to produce good quality academic texts but can also produce ‘hallucinations’ of entirely fictitious ideas pre­ sented as facts (Babl & Babl, 2023 who asked the GenAI to write aca­ demic abstracts), although the quality of the output has been shown to be enhanced through an iterative interaction with the user (Yang et al., 2025, Cheng et al., 2”

    From If ChatGPT can do it, where is my creativity? generative AI boosts performance but diminishes experience in creative writing (2025) · cites this paper as Yang et al., 2025

    The Results (p. 10) and Discussion (p. 13) find that writers who revise GenAI suggestions produce higher-quality essays than those who accept suggestions unchanged. The study measures the quality of the finished, human-revised writing; it does not show that iterative interaction improves the quality of GenAI's own output. (LLM, conf=0.85)

    Experimental — accuracy not yet measured
  • Previous studies on GAI-assisted writing often categorize writers based on overall AI usage throughout the entire writing session, overlooking temporal dynamics.

    “However, these existing studies often have two primary shortcomings: (i) They generally categorized writers based on their overall AI usage throughout the entire writing session, overlooking the temporal dynamics of these behaviours as the progression of a writing session (studies by Cheng et al., 2024;Shibani et al., 2023;Wambsganss et al., 2023;Yang et al., 2025);”

    From Ink and algorithm: Exploring temporal dynamics in generative <scp>AI</scp> ‐assisted writing (2026) · cites this paper as Yang et al., 2025

    Section 3.2 (p. 6) classifies writers in this study by the overall frequency of AI-assisted behavior patterns, supporting that part of the claim. But Section 2.2 (p. 5) describes an earlier study that used time-series clustering, so this paper does not support treating the neglect of temporal dynamics as a general shortcoming of previous studies. (LLM, conf=0.83)

    Experimental — accuracy not yet measured
  • Yang et al. (2025) demonstrated that writers who actively engaged in higher order cognitive processes consistently produced higher-quality essays.

    “Similarly, Yang et al. (2025) demonstrated that writers who actively engaged in higher order cognitive processes consistently produced higher-quality essays.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 8

  1. [1][1] R. Abdelghani, H. Sauzéon, and P.-Y. Oudeyer. 2023. Generative AI in the Classroom: Can Students Remain Active Learners? arXiv preprint arXiv:2310.03192 (2023). Manuscript submitted to ACM - · arXiv
  2. [3][3] N. Alipourfard, P. Fennell, and K. Lerman. 2018. Using Simpson's paradox to discover interesting patterns in behavioral data. In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 12. 2-11. -
  3. [14][14] A. Coenen, L. Davis, D. Ippolito, E. Reif, and A. Yuan. 2021. Wordcraft: A human-AI collaborative editor for story writing. arXiv preprint arXiv:2107.07430 (2021). - · arXiv
  4. [30][30] M. J. Ha. 2022. Syntactic complexity in EFL writing: Within-genre topic and writing quality. Computer-Assisted Language Learning Electronic Journal 23, 1 (2022), 187-205. -
  5. [34][34] Y. Jin, L. Yan, V. Echeverria, D. Gašević, and R. Martinez-Maldonado. 2024. Generative AI in Higher Education: A Global Perspective of Institutional Adoption Policies and Guidelines. arXiv preprint arXiv:2405.11800 (2024). - · arXiv
  6. [55][55] K. Sengupta, R. Maher, D. Groves, and C. Olieman. 2021. GenBiT: measure and mitigate gender bias in language datasets. Microsoft Journal of Applied Research 16 (2021), 63-71. -
  7. [56][56] A. Sharma and E. Kiciman. 2020. DoWhy: An End-to-End Library for Causal Inference. arXiv preprint arXiv:2011.04216 (2020). - · arXiv
  8. [64][64] K. Yang, Y. Cheng, L. Zhao, M. Raković, Z. Swiecki, D. Gašević, and G. Chen. 2024. Ink and Algorithm: Exploring Temporal Dynamics in Human-AI Collaborative Writing. arXiv:2406.14885 [cs.HC] https://arxiv.org/abs/2406.14885 - · arXiv

Cited by

Sorted by citations

Authors
Yang, K., Raković, M., Liang, Z., Yan, L., Zeng, Z., Fan, Y., Gašević, D., & Chen, G.
Affiliations
  • Monash University (AU)
  • Peking University (CN)
Venue
Proceedings of the 15th International Learning Analytics and Knowledge Conference
Publisher
ACM
Published
2025-03-03
Pages
568-578
OpenAlex
W4407842322
Cited by
26
References
55
Citations by year
2026: 11 · 2025: 15
FWCI
29.73
Type
conference paper
Language
en
Open access
Locations
  • International Conference on Learning Analytics and Knowledge (LAK) (publishedVersion, cc-by)
  • Monash University Research Portal (Monash University) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • HAL (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Australian Research Council (DP240100069, DP220101209)
  • Jacobs Foundation (CELLA 2 CERES)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Education › Writing and Handwriting Education
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
Keywords
essay quality, cognitive engagement, lexical sophistication, syntactic complexity, textual cohesion, language bias, writing assessment, higher-order cognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review