CoChat

NIRVANA: A Comprehensive Dataset for Reproducing How Students Use Generative AI for Essay Writing

Open access

The DOI matches its DataCite record; 49 of the 49 references it cites were found in a registry or an index; none of the 10 sources its claims cite is checked for support yet.

Abstract

With the rapid adoption of AI writing assistants in education, educators and researchers need empirical evidence to understand the impact on student writing and inform effective pedagogical design. Despite widespread use, we lack systematic understanding of how students engage with these tools during authentic writing tasks: when they seek assistance, what they ask, and how they incorporate AI-generated content into their essays. This gap limits evidence-based policy development and rigorous evaluation of generative AI's learning effects. To address this gap, we introduce NIRVANA, a dataset capturing how university students use generative AI while writing an analytical essay. The dataset includes 77 students who completed an essay task with access to ChatGPT, recording keystroke-level writing behavior, full ChatGPT conversation histories, and all text copied from ChatGPT, enabling a complete reconstruction of the writing process and revealing how AI assistance shapes student work. Our analysis identifies key behavioral patterns, including variation in ChatGPT query frequency and its relationship to essay characteristics such as length and readability. We identify four writing profiles based on students' contribution and revision patterns: Lead Authors, Collaborators, Drafters, and Vibe Writers. To support deeper investigation, we developed a replay interface that reconstructs the writing process; qualitative analysis of sampled replays demonstrates how this tool enables systematic examination of student-AI interactions.

Overview

CoChat Summary

TL;DR

NIRVANA is a dataset of 77 U.S. college students writing an analytical essay with an embedded in-house ChatGPT (GPT-3.5-turbo), capturing keystroke-level writing, full ChatGPT conversations, and copy/paste events. The paper reports exploratory quantitative patterns and four writer profiles, plus a replay interface for qualitative process analysis.

Key findings

  • Across 77 participants, essays averaged 382 words (σ = 235; median = 344), task time averaged 22.5 minutes (σ = 15.10; median = 20.6), ChatGPT queries averaged 4.51 (σ = 5.11; median = 2; skew = 2.24), and Dale-Chall grade level averaged 9.96 (σ = 1.34; median = 10.07).
  • Query count was positively correlated with word count (ρ = 0.485, p < 0.001), time taken (ρ = 0.493, p < 0.001), and Dale-Chall readability (ρ = 0.380, p < 0.001); no significant correlations were found with perceived ownership or the Creativity Support Index.
  • K-means clustering on Human Contribution Ratio (HCR) and Human Edit Ratio (HER) yielded four profiles: Lead Authors (n = 37; HCR = 0.98, HER = 0.95), Collaborators (n = 15; HCR = 0.53, HER = 0.62), Vibe Writers (n = 14; HCR = 0.01, HER = 0.06), and Drafters (n = 11; HCR = 0.09, HER = 0.64).
  • Cluster differences were significant for query count, H(3) = 9.016, p = 0.0277 (Lead Authors vs Collaborators p = 0.0141); time spent, H(3) = 11.69, p = 0.009 (Vibe Writers shorter than Lead Authors, Collaborators, and Drafters); Dale-Chall, H(3) = 32.26, p < 0.001 (Lead Authors lower than the other three groups); and perceived ownership, H(3) = 23.80, p < 0.001 (Lead Authors higher than Collaborators and Vibe Writers); CSI showed no significant differences.

Methods

  • A custom writing environment combined a text editor with an in-house ChatGPT interface using the OpenAI API and GPT-3.5-turbo; it logged keystrokes, cursor movements, insertions, deletions, copy-paste actions, and ChatGPT queries/responses with timestamps.
  • 77 U.S. college students were recruited via university mailing lists and Prolific; they completed pre/post surveys (TAM, SEWS, Perceived Ownership, CSI) and an ACT-style argumentative essay prompt about automation replacing human labor, advised to spend approximately 30 minutes.
  • Analyses included descriptive statistics, Spearman rank correlations between query count and writing outcomes, HCR/HER metrics, K-means clustering with silhouette/elbow selection (K = 4), Kruskal-Wallis tests with Dunn’s post hoc Bonferroni correction, and qualitative replay case studies.

Limitations

  • The paper states: a single ACT-style argumentative prompt limits generalizability; GPT-3.5-turbo may differ from more advanced models; the ~30-minute task does not capture longitudinal writing; the task was not graded and had no real academic stakes; recruitment via mailing lists and Prolific introduces self-selection bias; findings are exploratory and correlational, not causal.
  • The paper also states metric limitations: HCR overlooks structure and ideas when ChatGPT is used for revision, HER does not capture cognitive effort in planning, ideation, comprehending AI content, or deciding whether to incorporate it, and clustering may miss conceptual AI use (e.g., P26 was classified as a Collaborator but qualitatively resembled an editor of AI-generated content).
  • Evident from the text: the sample is limited to U.S. college students and one essay genre, and the study did not assess overall writing quality beyond readability metrics or trace semantic provenance of AI-generated ideas.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2604.07344

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 A screenshot of the Editor Interface
Figure 1 A screenshot of the Editor InterfaceCC-BY · From the PDF
Image 2
CC-BY · From the PDF
Image 3
CC-BY · From the PDF
Image 4
CC-BY · From the PDF
Figure 5 Five Clusters of Participants based on HCR and HER. The size of the marker indicates the number of participants with the same value pairs
Figure 5 Five Clusters of Participants based on HCR and HER. The size of the marker indicates the number of participants with the same value pairsCC-BY · From the PDF
Figure 6 Inquiry usage across clusters
Figure 6 Inquiry usage across clustersCC-BY · From the PDF
Image 7
CC-BY · From the PDF
Image 8
CC-BY · From the PDF
Figure 8 Dale-Chall scores across clusters
Figure 8 Dale-Chall scores across clustersCC-BY · From the PDF
Figure 10 Writing page of our application P26's case
Figure 10 Writing page of our application P26's caseCC-BY · From the PDF
Figure 11 SSE for our clusters
Figure 11 SSE for our clustersCC-BY · From the PDF
Figure 12 A screenshot of in-house ChatGPT provided to participants
Figure 12 A screenshot of in-house ChatGPT provided to participantsCC-BY · From the PDF

Verification

Claims the paper makes

0 Valid · 0 Not valid · 8 Not checked · 108 Own results and interpretations

32 more claims

Create a free account to see all 116 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 52
  • Probably not a substantial claim. 42
  • The quote only partly states this claim. 3

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Cited works that were retracted

Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.

1 of 1

Citations

Cites

1 retracted

Show all 49 references

Cited by

See all 1

Metadata

Authors
Jelson, A., Manesh, D., Lee, S., Jang, A., Dunlap, D., Maddox, T., Kim, Y.-H., & Lee, S. W.
Affiliations
  • Virginia Tech (US)
  • George Mason University (US)
  • NAVER Cloud (South Korea) (KR)
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
OpenAlex
W7152541924
Cited by
1
References
46
Citations by year
2026: 1
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Social Sciences › Education › Academic integrity and plagiarism
Keywords
generative AI in education, AI writing tools, ChatGPT, student essay writing, student-AI interaction, query frequency, readability, university students

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: NIRVANA: A Comprehensive Dataset for Reproducing How Students Use Generative AI for Essay Writing. https://app.cochat.ai/papers/10.48550/arxiv.2604.07344
The paper
Jelson, A., Manesh, D., Lee, S., Jang, A., Dunlap, D., Maddox, T., Kim, Y.-H., & Lee, S. W. (2026). NIRVANA: A Comprehensive Dataset for Reproducing How Students Use Generative AI for Essay Writing. arXiv. https://doi.org/10.48550/arxiv.2604.07344

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.