CoChat

Dense and Disconnected: Analyzing the Sedimented Style of ChatGPT-Generated Text at Scale

The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.

Abstract

ChatGPT and other LLMs are at the forefront of pedagogical considerations in classrooms across the academy. Many studies have spoken to the technology’s capacity to generate one-off texts in a variety of genres. This study complements those by inquiring into its capacity to generate compelling texts at scale. In this study, we quantitatively and qualitatively analyze a small corpus of generated texts in two genres and gauge it against novice and published academic writers along known dimensions of linguistic variation. Theoretically, we position and historicize ChatGPT as a writing technology and consider the ways in which generated text may not be congruent with established trajectories of writing development in higher education. Our study found that generated texts are more informationally dense than authored texts and often read as dialogically closed, “empty,” and “fluffy.” We close with a discussion of potentially explanatory linguistic features, as well as relevant pedagogical implications.

Overview

Integrity signals

Registry

Crossref · 10.1177/07410883241263528

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 49 references

Research works printed but not linked: 13

  1. Aull L., 2020, How students write
  2. Byrd A. (2023). Truth-telling: Critical inquiries on LLMs and the corpus texts that train them. Composition Studies, 51(1), 135–142. https://compositionstudiesjournal.files.wordpress.com/2023/06/byrd.pdf
  3. 10.1177/0741088317753348 · DOI
  4. Conrad D., 2001, Variation in English
  5. Devitt A. J., 2008, Writing genres
  6. Firth J. R., 1957, Papers in linguistics, 1934-1951
  7. Giltrow J., 2021, Academic writing
  8. Halliday M. A. K., 1976, Cohesion in English
  9. Radford A., 2018, Improving language understanding with unsupervised learning
  10. 10.58680/ccc19991338 · doi:10.58680/ccc19991338 · Paper page
  11. S. V. G. (1908). Do your pupils use erasers? Western Teacher: Devoted to Schoolroom Methods. Practical Aids and Usable Materials for Progressive Teachers, 16(5), 175–176. https://books.google.com/books?id=CKOfn_EPJNgC
  12. Vee A. (2023). Large language models write answers. Composition Studies, 51(1), 176–181. https://compositionstudiesjournal.files.wordpress.com/2023/06/vee.pdf
  13. Weaver W., 1949, Translation

Cited by

See all 47

Metadata

Authors
Markey, B., Brown, D. W., Laudenbach, M., & Kohler, A.
Affiliations
  • Carnegie Mellon University (US)
  • New Jersey Institute of Technology (US)
Corresponding author
Markey, Ben
Venue
Written Communication
Published
2024-08-04
Volume
41
Issue
4
Pages
571-600
ISSN
  • 0741-0883 (ISSN-L)
  • 1552-8472
OpenAlex
W4401316021
Cited by
27
References
49
Citations by year
2026: 12 · 2025: 15
FWCI
122.74
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
Written Communication (publishedVersion)
Topics
  • Social Sciences › Linguistics and Language › Discourse Analysis in Language Studies
  • Social Sciences › Education › Artificial Intelligence in Education
  • Social Sciences › Education › Linguistics and Education Research
Keywords
ChatGPT, large language models, AI-generated text, academic writing, writing development, information density, language variation, novice writers, writing instruction, college writing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Dense and Disconnected: Analyzing the Sedimented Style of ChatGPT-Generated Text at Scale. https://app.cochat.ai/papers/10.1177/07410883241263528
The paper
Markey, B., Brown, D. W., Laudenbach, M., & Kohler, A. (2024). Dense and Disconnected: Analyzing the Sedimented Style of ChatGPT-Generated Text at Scale. Written Communication. https://doi.org/10.1177/07410883241263528

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.