Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index; 4 of the 9 sources its claims cite were checked for support.

Human-AI collaboration patterns in AI-assisted academic writing

Open access
Nguyen, Andy, Hong, Yvonne, Dang, Belle, Huang, Xiaoshan · 2024 · Studies in Higher Education

Abstract

Artificial Intelligence (AI) has increasingly influenced higher education, notably in academic writing where AI-powered assisting tools offer both opportunities and challenges.Recently, the rapid growth of generative AI (GAI) has brought its impacts into sharper focus, yet the dynamics of its utilisation in academic writing remain largely unexplored.This paper focuses on examining the nature of human-AI interactions in academic writing, specifically investigating the strategies doctoral students employ when collaborating with a GAI-powered assisting tool.This study involves 626 recorded activities on how ten doctoral students interact with GAI-powered assisting tool during academic writing.AIdriven learning analytics approach was adopted for three layered analyses: (1) data pre-processing and analysis with quantitative content analysis, (2) sequence analysis with Hidden Markov Model (HMM) and hierarchical sequence clustering, and (3) pattern analysis with process mining.Findings indicate that doctoral students engaging in iterative, highly interactive processes with the GAI-powered assisting tool generally achieve better performance in the writing task.In contrast, those who use GAI merely as a supplementary information source, maintaining a linear writing approach, tend to get lower writing performance.This study points to the need for further investigations into human-AI collaboration in learning in higher education, with implications for tailored educational strategies and solutions.

Overview

CoChat Summary

TL;DR

This study used process mining and Hidden Markov Models on screen-recorded writing sessions to examine how 10 doctoral students collaborated with ChatGPT in a 30-minute academic writing task. It identified two strategy types—Structured Adaptivity and Unstructured Streamline—with the more iterative, interactive type achieving higher writing performance.

Key findings

  • HMM identified 3 hidden states: Content Pasting (14.6% of writing-tactic data; 82.6% paste-related actions), Content Copying (14.5%; 97.8% copying-related actions), and Component Shaping (70.8%; broader tactics such as AddReference, TypeContent, EditContent, PromptContent, PromptFollowUp).
  • Sequence clustering yielded two writing strategy types: Type 1 Structured Adaptivity (5 sequences) and Type 2 Unstructured Streamline (5 sequences), with a silhouette score of 0.372 and agglomerative coefficient of 0.77.
  • Type 1 had higher mean performance (M = 79.75, SD = 20.64) than Type 2 (M = 54.75, SD = 10.77; table reports 54.7), a significant difference (t = 2.4011, df = 6.0267, p ≤ 0.05). Type 1 used more PromptFollowUp (21 vs. 9) and EditPrompt (5 vs. 2); Type 2 had DeletePreviouslyPastedGeneratedContent (8 vs. 0).
  • Type 1 showed iterative, multifaceted engagement with ChatGPT; Type 2 used a more linear prompt-copy-paste approach with limited use and familiarity, associated with lower performance.

Methods

  • Participants were 10 doctoral students from Finland and New Zealand enrolled in English doctoral programs in Information Systems or Learning and Educational Technology. The online experiment via Zoom required a ~500-word essay on AI use in education within 30 minutes, allowing any tools including ChatGPT and Google Scholar; screen recording with audio and a pre-survey were used.
  • Data included 626 coded events from quantitative content analysis. The three-layer AI-driven learning analytics approach comprised: (1) content analysis, (2) Hidden Markov Model and Agglomerative Hierarchical Clustering, and (3) process mining using Fuzzy Miner and Fluxicon Disco. HMM state number was selected via AIC/BIC; writing was scored by a university lecturer using a rubric (content 30%, analysis 30%, organisation/structure 15%, quality 15%, word limit/referencing 10%).
  • Normality was checked with Shapiro-Wilk (W = 0.91534, p > 0.05), and a t-test compared mean performance scores between the two clusters.

Limitations

  • Stated: the small cohort (N = 10) limits statistical power, generalizability, and robustness; it heightens risk of sampling bias, and findings should be viewed as exploratory, requiring larger and more diverse samples.
  • Stated: variability in factors influencing academic writing performance (e.g., interest in the task, familiarity with subject matter) may affect outcomes and generalizability; the common assessment procedure may not fully capture the transformational potential of generative AI, focusing instead on efficiency gains.
  • Evident from the text: the study uses a single 30-minute task with one GAI tool (ChatGPT) in a specific doctoral population, and the data are not publicly available due to privacy or ethical restrictions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1080/03075079.2024.2323593

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Image 1
CC-BY · From the PDF
Image 2
CC-BY · From the PDF
Image 3
CC-BY · From the PDF
Image 4
CC-BY · From the PDF
Figure 1 The cognitive process of writing (Flowers and Hayes 1981).
Figure 1 The cognitive process of writing (Flowers and Hayes 1981).CC-BY · From the PDF
Image 6
CC-BY · From the PDF
Image 7
CC-BY · From the PDF
Image 8
CC-BY · From the PDF
Figure 4 Index plots depicting the hidden states characteristics for the two cluster types.
Figure 4 Index plots depicting the hidden states characteristics for the two cluster types.CC-BY · From the PDF
Figure 5 The sequence of writing tactics among the Type 1 - Structured Adaptively.
Figure 5 The sequence of writing tactics among the Type 1 - Structured Adaptively.CC-BY · From the PDF
Figure 6 The sequence of the writing tactics among the Type 2 - Unstructured Streamline.
Figure 6 The sequence of the writing tactics among the Type 2 - Unstructured Streamline.CC-BY · From the PDF

Verification

Claims the paper makes

3 Valid · 0 Not valid · 2 Not checked · 76 Own results and interpretations

35 more claims

Create a free account to see all 81 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 23
  • It may state two separate claims as one. 23
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 38 references

Research works printed but not linked: 1

  1. Kishore, S., Y. Hong, Andy Nguyen, and Saima. Qutab. 2023. “Should ChatGPT be Banned at Schools? Organising Visions for Generative Artificial Intelligence (AI) in Education.” International Conference on Information Systems (ICIS).

Cited by

See all 449

Metadata

Authors
Nguyen, A., Hong, Y., Dang, B., & Huang, X.
Affiliations
  • University of Oulu (FI)
  • Victoria University of Wellington (NZ)
  • McGill University (CA)
Corresponding author
Nguyen, Andy
Venue
Studies in Higher Education
Published
2024-02-28
Volume
49
Issue
5
Pages
847-864
ISSN
  • 0307-5079 (ISSN-L)
  • 1470-174X
OpenAlex
W4392239564
Cited by
449
References
38
Citations by year
2026: 250 · 2025: 170 · 2024: 29
FWCI
168.07
Type
journal article
Language
en
Open access
Locations
  • Studies in Higher Education (publishedVersion, cc-by)
  • Figshare (submittedVersion, cc-by-nc-nd)
  • Victoria University of Wellington
Funders
  • Research Council of Finland (350249)
  • University of Oulu
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Social Sciences › Education › Doctoral Education Challenges and Solutions
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
Keywords
human-AI collaboration, generative AI, AI-assisted writing, academic writing, doctoral students, writing performance, writing strategies, higher education, learning analytics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Human-AI collaboration patterns in AI-assisted academic writing. https://app.cochat.ai/papers/10.1080/03075079.2024.2323593
The paper
Nguyen, A., Hong, Y., Dang, B., & Huang, X. (2024). Human-AI collaboration patterns in AI-assisted academic writing. Studies in Higher Education. https://doi.org/10.1080/03075079.2024.2323593

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.