The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index; 13 of the 26 sources its claims cite were checked for support.
Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking
Open accessAbstract
Large language models are transforming the creative process by offering unprecedented capabilities to algorithmically generate ideas. While these tools can enhance human creativity when people co-create with them, it’s unclear how this will impact unassisted human creativity. We conducted two large pre-registered parallel experiments involving 1,100 participants attempting tasks targeting the two core components of creativity, divergent and convergent thinking. We compare the effects of two forms of large language model (LLM) assistance—a standard LLM providing direct answers and a coach-like LLM offering guidance—with a control group receiving no AI assistance, and focus particularly on how all groups perform in a final, unassisted stage. Our findings reveal that while LLM assistance can provide short-term boosts in creativity during assisted tasks, it may inadvertently hinder independent creative performance when users work without assistance, raising concerns about the long-term impact on human creativity and cognition.
Overview
CoChat Summary
TL;DR
Two pre-registered randomized experiments with 1,100 participants tested whether standard LLM answers or coach-like LLM guidance affect later unassisted divergent and convergent thinking. LLM assistance helped during assisted exposure rounds but did not improve—and sometimes worsened—independent performance afterward, with persistent idea homogenization in the strategy-guidance condition.
Key findings
- In divergent thinking (AUT), exposure to LLM assistance did not improve later unassisted originality or fluency; participants exposed to the List of Strategies performed significantly worse on test-round originality than those with no LLM exposure (p = 0.009), and test-round fluency did not differ significantly across conditions (H(2) = 1.14, p = 0.566).
- In convergent thinking (RAT), LLM Answer users did better during exposure, but in unassisted test rounds LLM Guidance participants performed significantly worse than no-LLM participants (p = 0.005; mean difference = -10.6, 95% CI [-18.6, -2.7]); LLM Answer did not differ significantly from no LLM (p = 0.682).
- Homogenization persisted for LLM-generated strategies: in the AUT test phase, List of Strategies participants generated more similar ideas than no-LLM participants (p = 0.017), while List of Ideas did not show the same lasting effect.
- Across both experiments, self-reported creativity declined after the experiment; in the RAT study, LLM Guidance showed an average reduction more than twice that of no LLM assistance, and no-LLM participants were most likely to find exposure rounds helpful.
Methods
- Two pre-registered between-subjects experiments recruited 1,100 Prolific participants (US/UK, fluent English): Experiment 1 had 460 participants for divergent thinking using the Alternate Uses Test (AUT), and Experiment 2 had 640 participants for convergent thinking using the Remote Associates Test (RAT).
- Both used exposure rounds with randomized conditions—Exp. 1: None, List of Ideas, List of Strategies; Exp. 2: None, LLM Answer, LLM Guidance—followed by a 1-minute Snake distractor and then unassisted test rounds.
- Exp. 1 analyzed 9,457 ideas using Kruskal-Wallis tests with Dunn’s post-hoc Bonferroni correction; originality was scored with a fine-tuned GPT-3 classifier (r = 0.81 with human judgments), and diversity used SBERT embeddings plus 150 Monte Carlo runs for group-level diversity. Exp. 2 used ANCOVA controlling for verbal fluency, followed by Tukey’s HSD.
Limitations
- Stated: creativity was measured only through short, verbal lab tasks; the short exposure period may yield smaller effects than real-world prolonged LLM use; only five AUT objects were used; LLM guidance was more verbose and introduced more cognitive load; mechanisms such as fatigue or dependency were not isolated; and the static LLM interaction may not reflect dynamic conversational AI use.
- Evident from the text: participants were Prolific crowdworkers in the US/UK who were fluent in English, so generalization to other populations and non-English contexts is uncertain; several pairwise comparisons were non-significant after correction, and the study measures only short-term residual effects after brief exposure.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures










Verification
Claims the paper makes
7 Valid · 0 Not valid · 8 Not checked · 103 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Across both experiments, we observed that LLM assistance was helpful during the exposure phase, aligning with previous research that highlights LLMs' ability to enhance creative task performance when users have access to AI-generated ideas or strategies [2,14].”
§ Human Creativity in the Age of LLMs Randomized Experiments on Divergent and Convergent Thinking > 5.1.1 LLMs Boost Performance During Exposure, but Unassisted Participants Excel in Test.
What the sources say
- Supportedcites [2] source
The citation supports the background point about LLM-assisted creative performance, not the citing authors’ observation about their own two experiments. In §4.3 (pp. 5–6), participants using ChatGPT generated more ideas and covered more idea categories than participants using the non-AI tool. (LLM, conf=0.88)
- Could not verifycites [14] sourceExperimental — accuracy not yet measured
The claim that assistance helped during the exposure phase is the citing authors’ own finding, not a finding attributed to this source. The cited paper’s Introduction (p. 1) says prior work has explored LLMs as creative-writing assistants, and §7.3 (p. 17) envisions future co-writing support, but neither reports that access to AI-generated ideas or strategies enhances users’ creative-task performance. Searches for that benefit and for an exposure phase found no supporting passage. (LLM, conf=0.70)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“We selected this task due to its widespread use in prior research [16,34], and its compatibility with LLMs.”
§ Human Creativity in the Age of LLMs Randomized Experiments on Divergent and Convergent Thinking > 4.1 Experimental Design
What the sources say
- Not checked — no full text of the sourcecites [16] source
- Partly supportedcites [34] sourceExperimental — accuracy not yet measured
The Introduction identifies the Remote Associates Task as an assessment that predominantly calls on convergent thinking, and the Methods describe its use in this study. The paper does not establish that the RAT was widely used in prior research; its statements that a task was often or extensively used refer to the Alternative Uses Task. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“Existing research has shown that LLM usage can lead to the homogenization of ideas within groups, where participants tend to converge on similar outcomes when using AIgenerated suggestions [2,19,45].”
§ Human Creativity in the Age of LLMs Randomized Experiments on Divergent and Convergent Thinking > 5.1.3 Persistent Homogenization of Ideas and the Challenges of Designing LLM Coaches.
What the sources say
- Supportedcites [2] source
The Discussion (p. 8) reports that ChatGPT-assisted ideas were significantly less semantically diverse at the group level than ideas produced with a non-AI tool. It attributes the effect to the LLM suggesting similar ideas to different users. (LLM, conf=0.93)
- Supportedcites [19] source
The paper’s Abstract reports that stories written with LLM-generated ideas were more similar to one another than stories written without them. The Discussion likewise says generative AI reduced the variance in creative outputs. This supports the claim that AI suggestions can lead participants’ outputs to converge. (LLM, conf=0.92)
- Supportedcites [45] source
The Introduction reports increased homogenization when writers used InstructGPT, and Section 5 says different users wrote similar essays on the same topic. This supports the hedged claim that LLM-generated suggestions can lead to convergence, though the paper did not find a statistically significant effect for the base GPT-3 model. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Interestingly, participants who received the List of Ideas performed slightly worse than other conditions, despite being exposed to LLM ideas with a mean originality of 3.25 (±0.1).”
§ Human Creativity in the Age of LLMs Randomized Experiments on Divergent and Convergent Thinking > 3.2.1 Originality.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is nearly 0.5 points higher than the average originality of ideas generated by participants in this condition (on a scale of 1 to 5).”
§ Human Creativity in the Age of LLMs Randomized Experiments on Divergent and Convergent Thinking > 3.2.1 Originality.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
63 more claims
Create a free account to see all 118 claims and every source verdict.
- It may state two separate claims as one. 28
- Probably not a substantial claim. 27
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“that experimental and clinical evidence documents accessdependent rather than transferable performance gains from AI assistance (Wiles et al., 2024;Budzyń et al., 2025;Kumar et al., 2025;Bangerl et al., 2025)”
From The Tragedy of the Cognitive Commons: How AI Could Disrupt the Regeneration of Professional Expertise (2026) · cites this paper as Kumar et al., 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Kumar et al. (2025) andBangerl et al. (2025) found limited skill carryover from AI-assisted ideation tasks, with the effect most pronounced on tasks requiring domain judgment.”
From The Tragedy of the Cognitive Commons: How AI Could Disrupt the Regeneration of Professional Expertise (2026) · cites this paper as Kumar et al. (2025)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Second, AI-assisted performance does not transfer to unassisted contexts.”
From The Tragedy of the Cognitive Commons: How AI Could Disrupt the Regeneration of Professional Expertise (2026) · cites this paper as Kumar et al. (2025)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This observation aligns with previous research indicating that LLMs may improve short-term creative performance but potentially impair independent creative ability over time [55].”
From Decoupling of Usefulness and Novelty: Evaluating the Impact of Generative AI on Design Outputs and Novice Designers' Creative Thinking (2026) · cites this paper as [55]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Emerging evidence shows measurable shifts in human writing styles, creative ideation, and divergent thinking following the widespread adoption of systems like ChatGPT [46, 6, 7, 24, 44, 79].”
From Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond) (2025) · cites this paper as [46]
Experimental — accuracy not yet measured
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(1, 1256) = 16.715reported p < .001 · recomputed p < .001
“F (1, 1256) = 16.715,p < .001”
- ConsistentF(2, 1256) = 5.134reported p = .006 · recomputed p = .006
“F (2, 1256) = 5.134, p = .006”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Human Creativity in the Age of LLMs Randomized Experiments on Divergent and Convergent Thinking > 3.1.3 Dependent Variables.
“We measure this by finding the maximum cosine similarity between the embedding of an idea in the Test set and all the ideas in the Exposure set.”
§ Human Creativity in the Age of LLMs Randomized Experiments on Divergent and Convergent Thinking > 3.2.3 Creative Flexibility.
“Figure 4c shows the average creative flexibility across conditions, which measures how different the ideas produced in the Test round are from those produced in the Exposure rounds (a higher value indicates more dissimilarity).”
Citations
Cites
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · cited 2×
“We calculated the median pairwise cosine distances between the embeddings of these ideas, using SBERT to encode them.”
Divergent Thinking as an Indicator of Creative Potential
Runco, M. A., & Acar, S. (2012) · Creativity Research Journal · doi:10.1080/10400419.2012.652929 · Cited by 1,427 · cited 4×
“Divergent thinking can be hypothesized to contribute primarily to P-creativity, where the generation of new ideas is the key [51–53].”
InfluentialGenerative artificial intelligence enhances creativity but reduces the diversity of novel content
Doshi, A., & Hauser, O. (2023) · doi:10.2139/ssrn.4535536 · Cited by 1,350 · cited 1×
“Existing research has shown that LLM usage can lead to the homogenization of ideas within groups, where participants tend to converge on similar outcomes when using AI-generated suggestions [2, 19, 45].”
In Praise of Convergent Thinking
Cropley, A. (2006) · Creativity Research Journal · doi:10.1207/s15326934crj1803_13 · Cited by 1,216 · cited 2×
“This is consistent with the idea that convergent thinking is more structured and goal-oriented, making it easier for participants to recognize when LLMs are effectively guiding them toward the correct solution [17, 42, 56].”
Autonomous chemical research with large language models
Boiko, D. A., MacKnight, R., Kline, B., & Gomes, G. (2023) · Nature · doi:10.1038/s41586-023-06792-0 · Cited by 1,050
Creativity: Yesterday, today and tomorrow
GUILFORD, J. P. (1967) · The Journal of Creative Behavior · doi:10.1002/j.2162-6057.1967.tb00002.x · Cited by 1,008 · cited 7×
“Convergent thinking, in contrast, focuses on refining these ideas to select the most effective solutions [4, 25].”
InfluentialArt and the science of generative AI
Epstein, Z., Hertzmann, A., the Investigators of Human Creativity, Akten, M., Farid, H., Fjeld, J., Frank, M. R., Groh, M., Herman, L., Leach, N., et al. (2023) · Science · doi:10.1126/science.adh4451 · Cited by 788 · cited 2×
“While there are concerns that this could lead to a deterioration of creative abilities, there is also optimism that these tools, if properly designed, could enhance human creativity while providing momentary assistance [20, 27].”
Normative data for 144 compound remote associate problems
Bowden, E. M., & Jung-Beeman, M. (2003) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195543 · Cited by 649 · cited 1×
“For the Remote Associates Test (RAT), we employed a widely-used dataset from Bowden and Jung-Beeman [11].”
Gilhooly, K. J., Fioratou, E., Anthony, S. H., & Wynn, V. (2007) · British Journal of Psychology · doi:10.1111/j.2044-8295.2007.tb00467.x · Cited by 643 · cited 1×
“We draw on key theories by Poincaré and Boden to frame our investigation [9, 21, 38].”
Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
(2025) · doi:10.1145/3706598 · Cited by 534
Research works printed but not linked: 3
- Kat Agres, Stephen McGregor, Matthew Purver, and Geraint A Wiggins. 2015. Conceptualizing Creativity: From Distributional Semantics to Conceptual Spaces.. In ICCC. 118–125.
- Mark A Runco. 1991. Divergent thinking. Ablex Publishing Corporation Norwood, NJ.
- Xinyi Wang. 2016. The Effect of Language Ability in Creativity Assessment. Ph. D. Dissertation.
Cited by
Student-AI collaborative creative Problem-Solving: The role of human agency
Guoa, W., Liang, Z., Wang, C., Li, X., Hu, H., Chen, S., Yu, Q., & Zhao, Q. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105433 · Cited by 51
Zhou, Y., Liu, Q., Huang, J., & Li, G. (2026) · Technology in Society · doi:10.1016/j.techsoc.2025.103087 · Cited by 24
CreAItive Collaboration? Users' Misjudgment of AI-Creativity Affects Their Collaborative Performance
Bangerl, M. M., Disch, L., David, T., & Pammer-Schindler, V. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713886 · Cited by 13
“A growing body of literature explores human-AI co-creation in divergent thinking tasks [ 47, 50, 63, 68, 88, 94, 108, 116].”
Desdevises, J. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1628486 · Cited by 32
Zheng, E., Fu, H., Thelwall, M. A., & Fang, Z. (2025) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.70028 · Cited by 13
“Persistent trust in model outputs could diminish researchers' agency and creativity in making independent critical evaluations (Kumar et al., 2025).”
Metadata
- Authors
- Kumar, H., Vincentius, J., Jordan, E., & Anderson, A.
- Affiliations
- University of Toronto (CA)
- Venue
- Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2025-04-25
- Pages
- 1-18
- OpenAlex
- W4409746216
- Cited by
- 129
- References
- 53
- Citations by year
- 2026: 93 · 2025: 36
- FWCI
- 306.19
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- University of Toronto
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- large language models, human creativity, divergent thinking, convergent thinking, AI-assisted creativity, pre-registered experiment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-12, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-12 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking. https://app.cochat.ai/papers/10.1145/3706598.3714198
- The paper
- Kumar, H., Vincentius, J., Jordan, E., & Anderson, A. (2025). Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3706598.3714198
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.