The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index; 4 of the 9 sources its claims cite were checked for support.
Human-AI collaboration patterns in AI-assisted academic writing
Open accessAbstract
Artificial Intelligence (AI) has increasingly influenced higher education, notably in academic writing where AI-powered assisting tools offer both opportunities and challenges.Recently, the rapid growth of generative AI (GAI) has brought its impacts into sharper focus, yet the dynamics of its utilisation in academic writing remain largely unexplored.This paper focuses on examining the nature of human-AI interactions in academic writing, specifically investigating the strategies doctoral students employ when collaborating with a GAI-powered assisting tool.This study involves 626 recorded activities on how ten doctoral students interact with GAI-powered assisting tool during academic writing.AIdriven learning analytics approach was adopted for three layered analyses: (1) data pre-processing and analysis with quantitative content analysis, (2) sequence analysis with Hidden Markov Model (HMM) and hierarchical sequence clustering, and (3) pattern analysis with process mining.Findings indicate that doctoral students engaging in iterative, highly interactive processes with the GAI-powered assisting tool generally achieve better performance in the writing task.In contrast, those who use GAI merely as a supplementary information source, maintaining a linear writing approach, tend to get lower writing performance.This study points to the need for further investigations into human-AI collaboration in learning in higher education, with implications for tailored educational strategies and solutions.
Overview
CoChat Summary
TL;DR
This study used process mining and Hidden Markov Models on screen-recorded writing sessions to examine how 10 doctoral students collaborated with ChatGPT in a 30-minute academic writing task. It identified two strategy types—Structured Adaptivity and Unstructured Streamline—with the more iterative, interactive type achieving higher writing performance.
Key findings
- HMM identified 3 hidden states: Content Pasting (14.6% of writing-tactic data; 82.6% paste-related actions), Content Copying (14.5%; 97.8% copying-related actions), and Component Shaping (70.8%; broader tactics such as AddReference, TypeContent, EditContent, PromptContent, PromptFollowUp).
- Sequence clustering yielded two writing strategy types: Type 1 Structured Adaptivity (5 sequences) and Type 2 Unstructured Streamline (5 sequences), with a silhouette score of 0.372 and agglomerative coefficient of 0.77.
- Type 1 had higher mean performance (M = 79.75, SD = 20.64) than Type 2 (M = 54.75, SD = 10.77; table reports 54.7), a significant difference (t = 2.4011, df = 6.0267, p ≤ 0.05). Type 1 used more PromptFollowUp (21 vs. 9) and EditPrompt (5 vs. 2); Type 2 had DeletePreviouslyPastedGeneratedContent (8 vs. 0).
- Type 1 showed iterative, multifaceted engagement with ChatGPT; Type 2 used a more linear prompt-copy-paste approach with limited use and familiarity, associated with lower performance.
Methods
- Participants were 10 doctoral students from Finland and New Zealand enrolled in English doctoral programs in Information Systems or Learning and Educational Technology. The online experiment via Zoom required a ~500-word essay on AI use in education within 30 minutes, allowing any tools including ChatGPT and Google Scholar; screen recording with audio and a pre-survey were used.
- Data included 626 coded events from quantitative content analysis. The three-layer AI-driven learning analytics approach comprised: (1) content analysis, (2) Hidden Markov Model and Agglomerative Hierarchical Clustering, and (3) process mining using Fuzzy Miner and Fluxicon Disco. HMM state number was selected via AIC/BIC; writing was scored by a university lecturer using a rubric (content 30%, analysis 30%, organisation/structure 15%, quality 15%, word limit/referencing 10%).
- Normality was checked with Shapiro-Wilk (W = 0.91534, p > 0.05), and a t-test compared mean performance scores between the two clusters.
Limitations
- Stated: the small cohort (N = 10) limits statistical power, generalizability, and robustness; it heightens risk of sampling bias, and findings should be viewed as exploratory, requiring larger and more diverse samples.
- Stated: variability in factors influencing academic writing performance (e.g., interest in the task, familiarity with subject matter) may affect outcomes and generalizability; the common assessment procedure may not fully capture the transformational potential of generative AI, focusing instead on efficiency gains.
- Evident from the text: the study uses a single 30-minute task with one GAI tool (ChatGPT) in a specific doctoral population, and the data are not publicly available due to privacy or ethical restrictions.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures











Verification
Claims the paper makes
3 Valid · 0 Not valid · 2 Not checked · 76 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This finding emphasises the potential benefits of adopting a cyclical approach to writing coordination with the GAI-powered tool (Dwivedi et al. 2023;Yan 2023).”
§ Human-AI collaboration patterns in AI-assisted academic writing > Discussion
What the sources say
- Partly supportedcites Dwivedi et al. 2023 sourceExperimental — accuracy not yet measured
Section 4.2.10.1 (p. 34) describes using ChatGPT to improve writing, and Section 4.2.7 (p. 30) discusses improving an AI-generated first draft. These support potential benefits from working with the tool, but the passages do not describe a cyclical approach to coordinating the writing process. (LLM, conf=0.84)
- Not checked — no full text of the sourcecites Yan 2023 source
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This state occupies 14.5% of the data on writing tactics”
§ Human-AI collaboration patterns in AI-assisted academic writing > RQ1a) What are the key hidden states in GAI-assisted academic writing? Hidden Markov model results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This state captures 14.6% of the total data on writing tactics”
§ Human-AI collaboration patterns in AI-assisted academic writing > RQ1a) What are the key hidden states in GAI-assisted academic writing? Hidden Markov model results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unlike the other two states, this one presents a broader range of writing tactics.”
§ Human-AI collaboration patterns in AI-assisted academic writing > RQ1a) What are the key hidden states in GAI-assisted academic writing? Hidden Markov model results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“our study shows that the lower-performing students demonstrate a more simplistic and potentially inefficient engagement with the GAI-powered tool.”
§ Human-AI collaboration patterns in AI-assisted academic writing > Discussion
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
35 more claims
Create a free account to see all 81 claims and every source verdict.
- Probably not a substantial claim. 23
- It may state two separate claims as one. 23
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“(ii) they revealed limited insight into writers' writing behaviours and the corresponding cognitive processes and thus offered limited implications for supporting human learning in this particular task (studies by Nguyen et al., 2024;Shibani et al., 2023;Wambsganss et al., 2023).”
From Ink and algorithm: Exploring temporal dynamics in generative AI ‐assisted writing (2026) · cites this paper as Nguyen et al., 2024
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Third, iterative AI-mediated revision cycles make the writing process more visible and deliberate—students compare AI-generated alternatives against their own drafts, evaluate competing formulations, and make explicit accept-or-reject decisions that strengthen metacognitive awareness of compositional choices (Chan et al., 2024; Hong et al., 2025; Nguyen et al., 2024).”
From From tool to scaffold: structured human–AI collaboration and its effects on academic writing and digital critical thinking among Saudi EFL learners (2026) · cites this paper as Nguyen et al., 2024
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Nguyen et al. (2024) proposed using Hidden Markov Models (HMM) and hierarchical sequence clustering to analyse human-AI interaction sequences in academic writing, finding that doctoral students who engaged in iterative, highly interactive processes with GAI-powered tools generally performed better in the writing task.”
From Ink and algorithm: Exploring temporal dynamics in generative AI ‐assisted writing (2026) · cites this paper as Nguyen et al., 2024
In the citing paper
“Nguyen et al. (2024) further show that human-AI collaboration in academic writing follows recurrent interactional patterns over time.”
From Identity, Agency, and Human–AI Mediation in L2 Academic Writing: An Interactional Study (2026) · cites this paper as Nguyen et al. (2024)
In the citing paper
“Writing, in this sense, is constituted through coordination between human and nonhuman actors whose contributions are asymmetrical but interdependent (Nguyen et al., 2024).”
From Identity, Agency, and Human–AI Mediation in L2 Academic Writing: An Interactional Study (2026) · cites this paper as Nguyen et al., 2024
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Human-AI collaboration patterns in AI-assisted academic writing > RQ1a) What are the key hidden states in GAI-assisted academic writing? Hidden Markov model results
“Based on the fitting scores, where the lowest values for BIC are preferred (see Table 2), our HMM model identified 3 hidden states in the sequence of writing tactics students employ while interacting with ChatGPT.”
§ Human-AI collaboration patterns in AI-assisted academic writing > Conclusions and final remarks
“| 2 | 4235.582 | 3902.631 | | 3 | 4307.197 | 3792.232 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Human-AI collaboration patterns in AI-assisted academic writing > RQ2. How is the effectiveness of each identified strategic pattern in GAI-assisted academic writing characterised by its specific nature and underlying attributes?
“Although both types expressed comparable confidence levels regarding digital skills (M = 4.40),”
§ Human-AI collaboration patterns in AI-assisted academic writing > Conclusions and final remarks
“| Digital Skills | 5.20 | 0.45 | 5.20 | 0.45 |”
Citations
Cites
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 2×
“…to aid in overcoming language and style-related challenges in complex academic writing, facilitating translation, vocabulary selection, sentence structuring, and adherence to scholarly tone, thereby enhancing the speed and clarity of the writing process (Dwivedi et al. 2023; Radford et al. 2021).”
BackgroundChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 3×
“Generative AI distinguishes itself from other technological advancements in its capacity to react based on prior interactions between human users and the AI system (Kasneci et al. 2023; Kishore et al. 2023).”
BackgroundDwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584
Cognitive load theory, learning difficulty, and instructional design
Sweller, J. (1994) · Learning and Instruction · doi:10.1016/0959-4752(94)90003-5 · Cited by 3,801 · cited 1×
“Moreover, writing processes share the working memory capacity (McCutchen 1996; 2000), and if the cognitive load surpasses this capacity, these processes can be hindered (Sweller 1994).”
BackgroundA Qualitative Framework for Collecting and Analyzing Data in Focus Group Research
Onwuegbuzie, A. J., Dickinson, W. B., Leech, N. L., & Zoran, A. G. (2009) · International Journal of Qualitative Methods · doi:10.1177/160940690900800301 · Cited by 1,775 · cited 1×
“To characterise the writing behaviour associated with the micro-processes undertaken by students, we conducted a qualitative content analysis utilising the constant comparison approach as delineated by Onwuegbuzie et al. (2009).”
MethodEthical principles for artificial intelligence in education
Nguyen, A., Ngo, H. N., Hong, Y., Dang, B., & Nguyen, B.-P. T. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11316-w · Cited by 1,228
Self-citationA cognitive process theory of writing
Flower, L., & Hayes, J. R. (1981) · College Composition & Communication · doi:10.58680/ccc198115885 · Cited by 1,025 · cited 2×
“Writing is a complex activity (Graham 2018), encompassing the navigation of language rules, communication norms, and the transition from the writer’s to the reader’s perspective.”
MethodLund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24750 · Cited by 919 · cited 1×
“…breakthroughs has been shown to enhance learning experiences, nurture creativity, and streamline knowledge management, there are growing concerns about their implications for academic integrity and the possible erosion of academic writing skills (Kasneci et al. 2023; Lund et al. 2023).”
BackgroundFuzzy Mining – Adaptive Process Simplification Based on Multi-perspective Metrics
Günther, C. W., & van der Aalst, W. M. P. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-75183-0_24 · Cited by 890 · cited 1×
“Specifically, our analysis was rooted in the Fuzzy Miner approach, as detailed by Günther and van der Aalst (2007), and utilised Fluxicon’s Disco analysis software, a recognised tool in process mining.”
Method
Research works printed but not linked: 1
- Kishore, S., Y. Hong, Andy Nguyen, and Saima. Qutab. 2023. “Should ChatGPT be Banned at Schools? Organising Visions for Generative Artificial Intelligence (AI) in Education.” International Conference on Information Systems (ICIS).
Cited by
Crawford, J., Allen, K.-A., Pani, B., & Cowling, M. (2024) · Studies in Higher Education · doi:10.1080/03075079.2024.2326956 · Cited by 224
“Some scholars are calling for more research into the AI-human collaboration (Gao et al., 2024; Nguyen et al., 2024).”
BackgroundPedagogical Applications of Generative AI in Higher Education: A Systematic Review of the Field
Qian, Y. (2025) · TechTrends · doi:10.1007/s11528-025-01100-1 · Cited by 206
“Similarly, the prominence of Writing/Composition (Cum-mings et al., 2024; Farazouli et al., 2024; Nguyen et al., 2024; Tsao and Nogues, 2024) and ESL/EFL studies (Esca-lante et al., 2024; Guo & Li, 2024; Guo et al., 2024; Waluyo & Kusumastuti, 2024; Wiboolyasarin et al., 2024) stems from GenAI’s…”
Generative AI: Is it a paradigm shift for higher education
O’Dea, X. (2024) · Studies in Higher Education · doi:10.1080/03075079.2024.2332944 · Cited by 151
Nguyen, A., Kremantzis, M., Essien, A., Petrounias, I., & Hosseini, S. (2024) · Journal of University Teaching and Learning Practice · doi:10.53761/caraaq92 · Cited by 146
“Recently, the widespread availability of GenAI, coupled with its advanced capabilities and ease of access and adoption, has introduced new affordances to teaching and learning (Enriquez et al., 2023; Nguyen et al., 2024).”
BackgroundInfluentialSelf-citationA qualitative systematic review on AI empowered self-regulated learning in higher education
Lan, M., & Zhou, X. (2025) · npj Science of Learning · doi:10.1038/s41539-025-00319-0 · Cited by 159
Metadata
- Authors
- Nguyen, A., Hong, Y., Dang, B., & Huang, X.
- Affiliations
- University of Oulu (FI)
- Victoria University of Wellington (NZ)
- McGill University (CA)
- Corresponding author
- Nguyen, Andy
- Venue
- Studies in Higher Education
- Publisher
- Informa UK Limited
- Published
- 2024-02-28
- Volume
- 49
- Issue
- 5
- Pages
- 847-864
- ISSN
- 0307-5079 (ISSN-L)
- 1470-174X
- OpenAlex
- W4392239564
- Cited by
- 449
- References
- 38
- Citations by year
- 2026: 250 · 2025: 170 · 2024: 29
- FWCI
- 168.07
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Studies in Higher Education (publishedVersion, cc-by)
- Figshare (submittedVersion, cc-by-nc-nd)
- Victoria University of Wellington
- Funders
- Research Council of Finland (350249)
- University of Oulu
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Education › Doctoral Education Challenges and Solutions
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Keywords
- human-AI collaboration, generative AI, AI-assisted writing, academic writing, doctoral students, writing performance, writing strategies, higher education, learning analytics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Human-AI collaboration patterns in AI-assisted academic writing. https://app.cochat.ai/papers/10.1080/03075079.2024.2323593
- The paper
- Nguyen, A., Hong, Y., Dang, B., & Huang, X. (2024). Human-AI collaboration patterns in AI-assisted academic writing. Studies in Higher Education. https://doi.org/10.1080/03075079.2024.2323593
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.