When ChatGPT is gone: Creativity reverts and homogeneity persists
Open accessAbstract
ChatGPT has been evidenced to enhance human performance in creative tasks. Yet, it is still unclear if this boosting effect sustains with and without ChatGPT. In a pre-registered seven-day lab experiment and a follow-up survey after 30 days of experiment completion, we examined the impacts of ChatGPT presence and absence on sustained creativity using a text dataset of 3302 creative ideas and 427 creative solutions from 61 college students. Participants in the treatment group used ChatGPT in creative tasks, while those in the control group completed the tasks by themselves. The findings show that although the boosting effect of ChatGPT was consistently observed over a five-day creative journey, human creative performance reverted to baseline when ChatGPT was down on the 7th and the 30th day. More critically, the use of ChatGPT in creative tasks resulted in increasingly homogenized contents, and this homogenization effect persisted even when ChatGPT was absence. These findings pose a challenge to the prevailing argument that ChatGPT can enhance human creativity. In fact, generative AI like ChatGPT lends to human with a temporary rise in creative performance but boxes human creative capability in the long run, highlighting the imperative for cautious generative AI integration in creative endeavors.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There is growing evidence that the introduction of LLM content in information ecosystems can contribute to output homogenization.
“There is growing evidence that the introduction of LLM content in information ecosystems can contribute to to output homogenization (Liu et al., 2024;Bommasani et al., 2022;Kleinberg & Raghavan, 2021).”
From Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews (2024) · cites this paper as Liu et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 19
- Acion, L., Rajngewerc, M., Randall, G., & Etcheverry, L. 2023. Generative AI poses ethical challenges for open science. Nature Human Behaviour, 7(11): 1800-1801.
- Bertolo, R., & Antonelli, A. 2024. Generative AI in scientific publishing: Disruptive or destructive? Nature Reviews Urology, 21(1): 1-2.
- Bockting, C. L., Van Dis, E. A. M., Van Rooij, R., Zuidema, W., & Bollen, J. 2023. Living guidelines for generative AI -why scientists must oversee its use. Nature, 622(7984): 693-696.
- Choudhury, M. 2023. Generative AI has a language problem. Nature Human Behaviour, 7(11): 1802-1803.
- Collaborative creativity in AI. 2022. Nature Machine Intelligence, 4(9): 733-733.
- H., et al. 2023. Art and the science of generative AI. Science, 380(6650): 1110-1111.
- Ghassemi, M., Birhane, A., Bilal, M., Kankaria, S., Malone, C., et al. 2023. ChatGPT one year on: Who is using it, how and why? Nature, 624(7990): 39-41.
- Grimes, M., Von Krogh, G., Feuerriegel, S., Rink, F., & Gruber, M. 2023. From Scarcity to Abundance: Scholars and Scholarship in an Age of Generative Artificial Intelligence. Academy of Management Journal, 66(6): 1617-1624.
- Jia, N., Luo, X., Fang, Z., & Liao, C. 2023. When and How Artificial Intelligence Augments Employee Creativity. Academy of Management Journal, amj.2022.0426.
- Kaiser, J. 2023. Funding agencies say no to AI peer review. Science, 381(6655): 261- 261.
- Kidd, C., & Birhane, A. 2023. How AI can distort human beliefs. Science, 380(6651): 1222-1223.
- Koivisto, M., & Grassini, S. 2023. Best humans still outperform artificial intelligence in a creative divergent thinking task. Scientific Reports, 13(1): 13601.
- Nakadai, R., Nakawake, Y., & Shibasaki, S. 2023. AI language tools risk scientific diversity and innovation. Nature Human Behaviour, 7(11): 1804-1805.
- Noorden, R. V., & Webb, R. n.d. 2023. ChatGPT and science: the AI system was a force in 2023-for good and bad. Nature, 624(7992), 509-509.
- Not a generative AI-generated Editorial. 2023. Nature Cancer, 4(2): 151-152.
- Porsdam Mann, S., Earp, B. D., Nyholm, S., Danaher, J., Møller, N., et al. 2023. Generative AI entails a credit-blame asymmetry. Nature Machine Intelligence, 5(5): 472-475.
- Reimers, N., & Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084. · arXiv
- Rafner, J., Beaty, R. E., Kaufman, J. C., Lubart, T., & Sherson, J. 2023. Creativity in the age of generative AI. Nature Human Behaviour, 7(11): 1836-1838.
- Van Noorden, R., & Perkel, J. M. 2023. AI and science: What 1,600 researchers think. Nature, 621(7980): 672-675.
Cited by
Sorted by citations
Liang, W., Izzo, Z., Zhang, Y., Lepp, H., Cao, H., Zhao, X., Chen, L., Ye, H., Liu, S., Huang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07183 · Cited by 73 · see citing works
- Authors
- Liu, Q., Zhou, Y., Huang, J., & Li, G.
- Affiliations
- Peking University (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2401.06816
- OpenAlex
- W4390961159
- Cited by
- 10
- References
- 0
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- ChatGPT, generative AI, human creativity, creative performance, homogenization, creative ideas, creative solutions, AI-assisted creativity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
