Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index; 13 of the 26 sources its claims cite were checked for support.

Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking

Open access
Kumar, Harsh, Vincentius, Jonathan, Jordan, Ewan, Anderson, Ashton · 2025 · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems

Abstract

Large language models are transforming the creative process by offering unprecedented capabilities to algorithmically generate ideas. While these tools can enhance human creativity when people co-create with them, it’s unclear how this will impact unassisted human creativity. We conducted two large pre-registered parallel experiments involving 1,100 participants attempting tasks targeting the two core components of creativity, divergent and convergent thinking. We compare the effects of two forms of large language model (LLM) assistance—a standard LLM providing direct answers and a coach-like LLM offering guidance—with a control group receiving no AI assistance, and focus particularly on how all groups perform in a final, unassisted stage. Our findings reveal that while LLM assistance can provide short-term boosts in creativity during assisted tasks, it may inadvertently hinder independent creative performance when users work without assistance, raising concerns about the long-term impact on human creativity and cognition.

Overview

CoChat Summary

TL;DR

Two pre-registered randomized experiments with 1,100 participants tested whether standard LLM answers or coach-like LLM guidance affect later unassisted divergent and convergent thinking. LLM assistance helped during assisted exposure rounds but did not improve—and sometimes worsened—independent performance afterward, with persistent idea homogenization in the strategy-guidance condition.

Key findings

  • In divergent thinking (AUT), exposure to LLM assistance did not improve later unassisted originality or fluency; participants exposed to the List of Strategies performed significantly worse on test-round originality than those with no LLM exposure (p = 0.009), and test-round fluency did not differ significantly across conditions (H(2) = 1.14, p = 0.566).
  • In convergent thinking (RAT), LLM Answer users did better during exposure, but in unassisted test rounds LLM Guidance participants performed significantly worse than no-LLM participants (p = 0.005; mean difference = -10.6, 95% CI [-18.6, -2.7]); LLM Answer did not differ significantly from no LLM (p = 0.682).
  • Homogenization persisted for LLM-generated strategies: in the AUT test phase, List of Strategies participants generated more similar ideas than no-LLM participants (p = 0.017), while List of Ideas did not show the same lasting effect.
  • Across both experiments, self-reported creativity declined after the experiment; in the RAT study, LLM Guidance showed an average reduction more than twice that of no LLM assistance, and no-LLM participants were most likely to find exposure rounds helpful.

Methods

  • Two pre-registered between-subjects experiments recruited 1,100 Prolific participants (US/UK, fluent English): Experiment 1 had 460 participants for divergent thinking using the Alternate Uses Test (AUT), and Experiment 2 had 640 participants for convergent thinking using the Remote Associates Test (RAT).
  • Both used exposure rounds with randomized conditions—Exp. 1: None, List of Ideas, List of Strategies; Exp. 2: None, LLM Answer, LLM Guidance—followed by a 1-minute Snake distractor and then unassisted test rounds.
  • Exp. 1 analyzed 9,457 ideas using Kruskal-Wallis tests with Dunn’s post-hoc Bonferroni correction; originality was scored with a fine-tuned GPT-3 classifier (r = 0.81 with human judgments), and diversity used SBERT embeddings plus 150 Monte Carlo runs for group-level diversity. Exp. 2 used ANCOVA controlling for verbal fluency, followed by Tukey’s HSD.

Limitations

  • Stated: creativity was measured only through short, verbal lab tasks; the short exposure period may yield smaller effects than real-world prolonged LLM use; only five AUT objects were used; LLM guidance was more verbose and introduced more cognitive load; mechanisms such as fatigue or dependency were not isolated; and the static LLM interaction may not reflect dynamic conversational AI use.
  • Evident from the text: participants were Prolific crowdworkers in the US/UK who were fluent in English, so generalization to other populations and non-English contexts is uncertain; several pairwise comparisons were non-significant after correction, and the study measures only short-term residual effects after brief exposure.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3706598.3714198

Cites

71 of 71 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Experimental framework for measuring the impact of AI use on Human creativity. Participants engage in a series of Exposure rounds where they are randomized to either receive - (A) No assistance, (B) LLM solution (standard): This could be analogous to using a chat LLM such as ChatGPT for the task, or (C) LLM guidance (coach-like): In this case, participants receive response from a customized LLM which guides them through the creative process. Finally, in the last round, all participants are asked to do the same creative task without any assistance as a Test . (D) The performance and creative outputs in this unassisted round are the primary measures for evaluating the impact of using LLMs on Human cognition.
Figure 1 Experimental framework for measuring the impact of AI use on Human creativity. Participants engage in a series of Exposure rounds where they are randomized to either receive - (A) No assistance, (B) LLM solution (standard): This could be analogous to using a chat LLM such as ChatGPT for the task, or (C) LLM guidance (coach-like): In this case, participants receive response from a customized LLM which guides them through the creative process. Finally, in the last round, all participants are asked to do the same creative task without any assistance as a Test . (D) The performance and creative outputs in this unassisted round are the primary measures for evaluating the impact of using LLMs on Human cognition.CC-BY · From the PDF
Figure 2 Schematic of design for Experiment 1 on divergent thinking.
Figure 2 Schematic of design for Experiment 1 on divergent thinking.CC-BY · From the PDF
Figure 3 Interface used in the divergent thinking experiment across all 3 exposure round conditions and test rounds.
Figure 3 Interface used in the divergent thinking experiment across all 3 exposure round conditions and test rounds.CC-BY · From the PDF
Figure 4 Plots of the Alternate Uses Task ideas for the various divergent thinking dimensions (Segmented by phase and/or LLM response type). The left figure shows idea originality scores, the middle figure indicates idea fluency, and the right figure presents participant creative flexibility.
Figure 4 Plots of the Alternate Uses Task ideas for the various divergent thinking dimensions (Segmented by phase and/or LLM response type). The left figure shows idea originality scores, the middle figure indicates idea fluency, and the right figure presents participant creative flexibility.CC-BY · From the PDF
Figure 5 Plots of the average individual- (left) and group-level (right) median diversity, segmented by experiment conditions and phases. Higher values denote more difference between ideas. Error bars represent ± one standard error of mean.
Figure 5 Plots of the average individual- (left) and group-level (right) median diversity, segmented by experiment conditions and phases. Higher values denote more difference between ideas. Error bars represent ± one standard error of mean.CC-BY · From the PDF
Figure 6 Plots of subjective measures collected before and after participants completed the Alternate Uses Task. The left figure shows participants' change in self perceived creativity levels (Based on how many % of humans they felt they were more creative than), the middle figure indicates how their feelings towards the increased use of AI computer programs in daily life changed (Between More concerned than excited/More excited than concerned/Equally excited and concerned), and the right figure presents how much difficulty they had in coming up with ideas for the test object, all segmented by the three LLM response types.
Figure 6 Plots of subjective measures collected before and after participants completed the Alternate Uses Task. The left figure shows participants' change in self perceived creativity levels (Based on how many % of humans they felt they were more creative than), the middle figure indicates how their feelings towards the increased use of AI computer programs in daily life changed (Between More concerned than excited/More excited than concerned/Equally excited and concerned), and the right figure presents how much difficulty they had in coming up with ideas for the test object, all segmented by the three LLM response types.CC-BY · From the PDF
Figure 7 Schematic of design for Experiment 2 on convergent thinking.
Figure 7 Schematic of design for Experiment 2 on convergent thinking.CC-BY · From the PDF
Figure 8 Interface used in the convergent thinking experiment across all 3 exposure round conditions and test rounds.
Figure 8 Interface used in the convergent thinking experiment across all 3 exposure round conditions and test rounds.CC-BY · From the PDF
Figure 9 Plot of participant accuracy on the Remote Associates Task, segmented by exposure and test phase across all experimental conditions.
Figure 9 Plot of participant accuracy on the Remote Associates Task, segmented by exposure and test phase across all experimental conditions.CC-BY · From the PDF
Figure 10 Plots of participant accuracy on the Remote Associates Task, segmented by exposure and test phases across all experimental conditions. The top left figure shows participants' change in self perceived creativity ratings (Based on how many % of humans they felt they were more creative than), the top right figure indicates how their feelings towards the increased use of AI computer programs in daily life changed (Between More concerned than excited/More excited than concerned/Equally excited and concerned), the bottom left figure displays how much difficulty they had in coming up with answers for the test phase, and the bottom right figure presents how many participants found the exposure rounds helpful.
Figure 10 Plots of participant accuracy on the Remote Associates Task, segmented by exposure and test phases across all experimental conditions. The top left figure shows participants' change in self perceived creativity ratings (Based on how many % of humans they felt they were more creative than), the top right figure indicates how their feelings towards the increased use of AI computer programs in daily life changed (Between More concerned than excited/More excited than concerned/Equally excited and concerned), the bottom left figure displays how much difficulty they had in coming up with answers for the test phase, and the bottom right figure presents how many participants found the exposure rounds helpful.CC-BY · From the PDF

Verification

Claims the paper makes

7 Valid · 0 Not valid · 8 Not checked · 103 Own results and interpretations

63 more claims

Create a free account to see all 118 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 28
  • Probably not a substantial claim. 27

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(1, 1256) = 16.715reported p < .001 · recomputed p < .001

    “F (1, 1256) = 16.715,p < .001”

  • ConsistentF(2, 1256) = 5.134reported p = .006 · recomputed p = .006

    “F (2, 1256) = 5.134, p = .006”

2 of 2

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 53 references

Research works printed but not linked: 3

  1. Kat Agres, Stephen McGregor, Matthew Purver, and Geraint A Wiggins. 2015. Conceptualizing Creativity: From Distributional Semantics to Conceptual Spaces.. In ICCC. 118–125.
  2. Mark A Runco. 1991. Divergent thinking. Ablex Publishing Corporation Norwood, NJ.
  3. Xinyi Wang. 2016. The Effect of Language Ability in Creativity Assessment. Ph. D. Dissertation.

Cited by

See all 129

Metadata

Authors
Kumar, H., Vincentius, J., Jordan, E., & Anderson, A.
Affiliations
University of Toronto (CA)
Venue
Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2025-04-25
Pages
1-18
OpenAlex
W4409746216
Cited by
129
References
53
Citations by year
2026: 93 · 2025: 36
FWCI
306.19
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
University of Toronto
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
  • Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords
large language models, human creativity, divergent thinking, convergent thinking, AI-assisted creativity, pre-registered experiment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-12, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-12 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking. https://app.cochat.ai/papers/10.1145/3706598.3714198
The paper
Kumar, H., Vincentius, J., Jordan, E., & Anderson, A. (2025). Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3706598.3714198

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.