Cited by 135Innovative Human-Technology InteractionArt, Technology, and CultureInformation Systems Theories and Implementation

Social Dynamics of AI Support in Creative Writing

Open access
Gero, Katy Ilonka, Long, Tao, Chilton, Lydia B·2023·Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Abstract

Recently, large language models have made huge advances in generating coherent, creative text. While much research focuses on how users can interact with language models, less work considers the social-technical gap that this technology poses. What are the social nuances that underlie receiving support from a generative AI? In this work we ask when and why a creative writer might turn to a computer versus a peer or mentor for support. We interview 20 creative writers about their writing practice and their attitudes towards both human and computer support. We discover three elements that govern a writer’s interaction with support actors: 1) what writers desire help with, 2) how writers perceive potential support actors, and 3) the values writers hold. We align our results with existing frameworks of writing cognition and creativity support, uncovering the social dynamics which modulate user responses to generative technologies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3544548.3580782

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Only a handful of user studies have investigated how professional writers perceive and use AI systems.

    “Yet, only a handful of user studies have investigated how professional writers perceive and use AI systems [13,27,28]”

    From Playwriting with Large Language Models: Perceived Features, Interaction Strategies and Outcomes (2024) · cites this paper as [27]

    The Methodology (section 3.1) describes interviews with 20 creative writers about their attitudes toward hypothetical AI writing tools and, for some participants, their use of SudoWrite. But Participant Population (section 3.2.4) says only two were full-time writers and the sample skewed slightly amateur, so this is not specifically a study of professional writers. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • Empirical studies describing how users evaluate LLMs are still a minority and difficult to generalise due to variation of participants, methods, systems, and disciplinary perspectives.

    “Empirical studies describing how users evaluate LLMs are still a minority and difficult to generalise due to a variation of participants, methods, systems, and disciplinary perspectives [13,27,28,38,47].”

    From Playwriting with Large Language Models: Perceived Features, Interaction Strategies and Outcomes (2024) · cites this paper as [27]

    Section 5.1.3 says responses to AI writing tools vary across users and that studies rarely account for this variation. Section 7.2 says the study’s participant population limits broader conclusions. The paper does not establish that empirical studies of how users evaluate LLMs are a minority, or attribute difficulty generalising across that literature to differences in methods, systems, and disciplinary perspectives. The indexed passages provide no page numbers. (LLM, conf=0.85)

    Experimental — accuracy not yet measured
  • Current LLM systems fail to satisfy writers' needs because they do not understand writers' intentions.

    “Current systems fail to satisfy the writers' needs because they do not understand their intentions [27].”

    From Playwriting with Large Language Models: Perceived Features, Interaction Strategies and Outcomes (2024) · cites this paper as [27]

    Section 6.2.3 says feedback often failed to meet writers’ needs when a support actor did not understand their intention, and that writers were skeptical computers could do so. It also says little work explicitly models writers’ intentions. The claim overstates these qualified observations as a categorical failure of current systems. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • LLMs are often described as biased, perpetuate stereotypes, and jeopardise trust in generated content.

    “The models are often described as biased.They perpetuate stereotypes and jeopardise trust in the generated content [27,38].”

    From Playwriting with Large Language Models: Perceived Features, Interaction Strategies and Outcomes (2024) · cites this paper as [27]

    Section 5.1.2 says language models reflect white, Western, and male perspectives shaped by their training data, supporting the bias portion of the claim. It also says making those assumptions explicit will not necessarily make writers see the models as trusted readers. The passages do not establish that the models perpetuate stereotypes or jeopardise trust in generated content specifically. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • Despite limitations, playwrights identified utility at different stages of creative writing, from brainstorming through the summarisation of relevant sources to editing.

    “Despite these limitations, playwrights identified utility at different stages of creative writing, from brainstorming through the summarisation of relevant sources to editing [27,42,47].”

    From Playwriting with Large Language Models: Perceived Features, Interaction Strategies and Outcomes (2024) · cites this paper as [27]

    The Methodology (§3.1) describes brainstorming and rewriting features, and Reviewing (§4.2.3) reports writers seeking feedback useful for editing. But the study interviewed creative writers across genres, not playwrights as a group, and our copy does not report that they found summarisation of relevant sources useful; the interview guide only asks about a hypothetical summarisation capability. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. [23]23. Ming Liu, Rafael A Calvo, and Vasile Rus. 2020. Automatic Generation and Ranking of Questions for Critical Review. (2020), 15.
  2. [28]28. Michael Quinn Patton. 2002. Qualitative Research and Evaluation Methods (Chapter 5) (3 ed.). Sage Publications.
  3. [41]41. Feiyu Xu, Hans Uszkoreit, Yangzhou Du, Wei Fan, Dongyan Zhao, and Jun Zhu. 2019. Explainable AI: A Brief Survey on History, Research Areas, Approaches and Challenges. In Natural Language Processing and Chinese Computing, Jie Tang, Min-Yen Kan, Dongyan Zhao, Sujian Li, and Hongying Zan (Eds.). Springer International Publishing, Cham, 563-574.

Cited by

Sorted by citations

Authors
Gero, K. I., Long, T., & Chilton, L. B.
Affiliations
Columbia University (US)
Venue
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2023-04-19
Pages
1-15
OpenAlex
W4366594764
Cited by
129
References
24
Citations by year
2026: 43 · 2025: 41 · 2024: 42 · 2023: 3
FWCI
29.94
Type
conference paper
Language
en
Open access
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion)
Funders
National Science Foundation (2129020)
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Social Sciences › Arts and Humanities › Visual Arts and Performing Arts › Art, Technology, and Culture
  • Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
Keywords
generative AI, creative writing, large language models, peer support, mentor support, writing cognition, creativity support

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review