The DOI matches its Crossref record; 95 of the 95 references it cites were found in a registry or an index; 17 of the 38 sources its claims cite were checked for support.
Decoupling of Usefulness and Novelty: Evaluating the Impact of Generative AI on Design Outputs and Novice Designers' Creative Thinking
Open accessAbstract
People are increasingly leveraging generative AI (GenAI) for design tasks, making it critical to understand GenAI’s impact on design outcomes and users’ creative capabilities. We conducted a within-subjects experiment where 36 participants designed advertisements both with and without GenAI. Evaluations from clients and online volunteers revealed that GenAI-supported designs were perceived as significantly more creative and unconventional. Additionally, online volunteers, but not clients, rated these designs as more visually appealing. However, neither group perceived differences in usefulness, and clients noted no improvement in brand alignment, highlighting a notable decoupling of novelty and usefulness (two established components of creativity) in GenAI-supported design outputs. Although short-term GenAI use did not broadly influence participants’ creative thinking or experience, subgroup analyses indicated increases in divergent thinking among participants new to GenAI relative to participants with GenAI experience. We discuss the implications of the decoupling effect and GenAI’s influence on humans’ creativity.
Overview
CoChat Summary
TL;DR
In a within-subjects lab experiment, 36 participants designed social media advertisements with and without generative AI support, and their outputs were rated by five client stakeholders and 155 blinded online volunteers. GenAI-supported designs were judged significantly more creative and unconventional, but not more useful, revealing a decoupling of novelty and usefulness in GenAI-supported design outputs.
Key findings
- Client evaluators rated GenAI-supported designs as significantly more creative (β = 0.805, SE = 0.183, Z = 4.405, p < .001) and more unconventional (β = 1.300, SE = 0.193, Z = 6.721, p < .001) than human-only designs, but found no significant differences in perceived usefulness (β = 0.019, SE = 0.163, Z = 0.119, p = .905), visual appeal (β = 0.264, p = .153), or brand alignment (β = −0.188, p = .359).
- Online volunteers likewise rated GenAI-supported designs as more creative (β = 0.900, SE = 0.149, Z = 6.040, p < .001), more unconventional (β = 0.658, SE = 0.106, Z = 6.218, p < .001), and more visually appealing (β = 0.447, SE = 0.138, Z = 3.245, p < .01), with no usefulness difference (β = 0.001, SE = 0.112, Z = 0.006, p = .995).
- No overall condition differences appeared in mood (relaxation, pleasure, energy), self-efficacy, divergent thinking (AUT), or convergent thinking (Anagram). In subgroup analysis, participants without prior GenAI design tool experience showed higher fluency (M = 14.333, SD = 4.384 vs. M = 12.500, SD = 4.573, Z = −2.112, p < .05) and flexibility (M = 12.250, SD = 3.562 vs. M = 10.167, SD = 3.131, Z = −2.203, p < .05) after the GenAI session; those with prior experience showed no significant differences.
- Of 105 eligible designs (52 human-only, 53 GenAI-supported), participants selected their own GenAI-supported design as "best" in 17 of 36 cases (47.2%). Client qualitative comments praised GenAI designs' bright colors and diversity but criticized them as "too busy," illegible, or socially inappropriate/uncanny.
Methods
- Study One: within-subjects lab experiment with 36 participants (94% under 30; 81% women; 67% with at least some prior GenAI design experience), each completing two counterbalanced 25-minute sessions — one using ChatGPT (DALL-E 3) and one using their own preferred tools — producing 110 designs (105 eligible). Measures: Alternative Uses Test (divergent thinking), Anagram Test (convergent thinking), and self-reported mood and self-efficacy; analyzed with Wilcoxon signed-rank tests.
- Study Two: five "client" evaluators (members of the target research group) rated all 105 designs (N = 525 ratings) on usefulness, creativity, unconventionality, brand alignment, and visual appeal, plus optional free-response comments; 155 online volunteers (LabintheWild, aged 9–64, M = 25.6) each rated 10 randomly presented designs (N = 1550 ratings), blinded to condition.
- Analysis: Generalized Linear Mixed Models with fixed effect of GenAI support and random intercepts for raters and for designs nested within participants; thematic analysis of client qualitative comments.
Limitations
- Stated by the authors: reliance on subjective rater evaluations rather than objective metrics such as click-through rates; use of only GPT-4/DALL-E 3 rather than design-specific tools, limiting generalizability; the artificial controlled lab setting; qualitative analysis drawn from optional free-response text from only five clients; and a sample of mostly novice/early-career designers from a college community and alumni networks, limiting generalizability to experienced industry designers.
- Evident in the text: the intervention was a single 25-minute session, so findings speak only to short-term effects; the client rater pool was very small (n = 5), and the study notes that knowing a design is AI-generated might itself influence evaluator ratings, which was not tested.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
10 Valid · 1 Not valid · 10 Not checked · 76 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“We also saw recurring patterns such as very bright colors and complex compositions, hinting at a converging “AI style,” which echoes concerns that generative AI homogenizes outputs [99] and reduces diversity at a collective level [25].”
§ 7 Discussion > 7.1 GenAI’s Impact on Decoupling Novelty vs. Usefulness of Design Outputs
What the sources say
- Partly supportedcites [99] [99] Veera Vimpari, Annakaisa Kultima, Perttu Hämäläinen, and Christian Guckelsberger. 2023. “An Adapt-or-Die Type of Situation”: Perception, Adoption, and Use of Text-to-Image-Generation AI by Game Industry Professionals. Proc. ACM Hum.-Comput. Interact. 7, CHI PLAY, Article 379 (Oct. 2023), 34 pages. Digital Library Google Scholar a [...] that GenAI may homogenize creative outputs b [...] that generative AI homogenizes outputs sourceExperimental — accuracy not yet measured
The cited paper reports participants’ concerns about a lack of true novelty in text-to-image outputs (Comparison to Related Work, p. 23), which partly supports the cited concern about homogenization. It does not establish that outputs converge on a shared visual style. The bright colors and complex compositions are the citing authors’ own observations, not findings attributed to this source. (LLM, conf=0.72)
- Supportedcites [25] [25] Anil R Doshi and Oliver P Hauser. 2024. Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances 10, 28 (2024), eadn5290. Google Scholar a [...] rather than self-assessed by creators b [...] begun to probe GenAI’s role in creativity c [...] conceptual diversity at a collective level d [...] ], fictional stories e [...] and reduces diversity at a collective level f [...] inadvertently homogenize design outcomes source
The Results section (‘Similarity of stories’; page unavailable) reports that stories written with access to generative AI ideas were more similar to other stories in the same condition. The Discussion says this could reduce collective novelty. The bright colors and complex compositions are the citing authors’ own observations, not findings attributed to this source. (LLM, conf=0.92)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Using GenAI in design also places metacognitive demands [94] on users, requiring sustained monitoring and control.”
§ 7 Discussion > 7.1 GenAI’s Impact on Decoupling Novelty vs. Usefulness of Design Outputs
What the sources say
- Supportedcites [94] [94] Lev Tankelevitch, Viktor Kewenig, Auste Simkute, Ava Elizabeth Scott, Advait Sarkar, Abigail Sellen, and Sean Rintel. 2024. The metacognitive demands and opportunities of generative AI. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–24. Go to Citation Digital Library Google Scholar source
Section 3 says that working with GenAI imposes extensive metacognitive monitoring and control demands, and names design among the domains affected. The paper describes these demands across prompting, evaluating outputs, and deciding how to incorporate GenAI into workflows. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that there were no significant differences between conditions in mood (relaxation, pleasure, energy), self-efficacy, inherent divergent thinking (AUT), or convergent thinking (Anagram).”
§ 4 Study One Results: Participants’ Experience > 4.1 Overall Effects on Participants’ Mood, Self-Efficacy, and Creativity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, clients also pointed out that some GenAI-supported designs misaligned with social norms and could unintentionally convey inappropriate messages.”
§ 6 Study Two Results: Design Product Evaluation > 6.3 Qualitative Analysis of Client Evaluation Comments
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, there were no significant condition differences in convergent thinking, mood, or self-efficacy regardless of prior GenAI experience.”
§ 4 Study One Results: Participants’ Experience > 4.2 Subgroup Analysis: Prior Experience with GenAI
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
51 more claims
Create a free account to see all 97 claims and to verify the 6 references still open.
- Probably not a substantial claim. 25
- It may state two separate claims as one. 25
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentz = -2.112reported p < .05 · recomputed p = .035
“Z = −2.112, p < .05”
- Consistentz = -2.203reported p < .05 · recomputed p = .028
“Z = −2.203, p < .05”
- Consistentz = 4.405reported p < .001 · recomputed p < .001
“Z = 4.405, p < .001”
- Consistentz = 6.721reported p < .001 · recomputed p < .001
“Z = 6.721, p < .001”
- Consistentz = 0.119reported p = .905 · recomputed p = .905
“Z = 0.119, p = .905”
6 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Braun, V., & Clarke, V. (2012) · APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. · doi:10.1037/13620-004 · Cited by 10,299
The standard definition of creativity
Runco, M. A., & Jaeger, G. J. (2012) · Creativity Research Journal · doi:10.1080/10400419.2012.650092 · Cited by 3,895 · cited 1×
“Creativity fundamentally requires both novelty and usefulness [61, 63, 73, 88].”
Creativity in the design process: co-evolution of problem–solution
Dorst, K., & Cross, N. (2001) · Design Studies · doi:10.1016/s0142-694x(01)00009-6 · Cited by 2,357
Plucker, J. A., Beghetto, R. A., & Dow, G. T. (2004) · Educational Psychologist · doi:10.1207/s15326985ep3902_1 · Cited by 1,794 · cited 1×
“…creativity’s significant role in improving problem-solving, fostering innovation, boosting productivity, and contributing to personal socio-emotional well-being, satisfaction, and life success [6, 76, 93, 96, 97], noting its significance for both individual development and societal advancement.”
Generative artificial intelligence enhances creativity but reduces the diversity of novel content
Doshi, A., & Hauser, O. (2023) · doi:10.2139/ssrn.4535536 · Cited by 1,350
The Cambridge Handbook of Creativity
(2010) · doi:10.1017/cbo9780511763205 · Cited by 1,291
AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
Gerlich, M. (2025) · Societies · doi:10.3390/soc15010006 · Cited by 1,248
AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
Gerlich, M. (2025) · doi:10.2139/ssrn.5082524 · Cited by 1,238
Creativity: Yesterday, today and tomorrow
GUILFORD, J. P. (1967) · The Journal of Creative Behavior · doi:10.1002/j.2162-6057.1967.tb00002.x · Cited by 1,008 · cited 2×
“Divergent thinking relates to the ability to generate multiple creative ideas, typically assessed via tests like the Alternative Uses Test (AUT) [35] and Torrance Tests of Creative Thinking [17, 95], which measures how creatively participants can think of non-common uses for common objects like a…”
Creativity support tools: accelerating discovery and innovation
Shneiderman, B. (2007) · Communications of the ACM · doi:10.1145/1323688.1323689 · Cited by 765 · cited 1×
“With the advancement of computer technology, modern software tools have been created to support creative processes, which have been classified broadly as creativity support tools (CSTs) [30, 41, 90, 91].”
Research works printed but not linked: 6
- Teresa M Amabile. 1996. Creativity and innovation in organizations. Vol. 5. Harvard Business School Boston.
- Oxford Analytica. 2023. GenAI will transform workplace tasks across industries. Emerald Expert Briefingsoxan-db (2023).
- Nicholas M Davis, Yanna Popova, Ivan Sysoev, Chih-Pin Hsiao, Dingtian Zhang, and Brian Magerko. 2014. Building Artistic Computer Colleagues with an Enactive Model of Creativity.. In ICCC. 38–45.
- Daniel Kahneman. 2011. Thinking, fast and slow. macmillan.
- Jie Li, Hancheng Cao, Laura Lin, Youyang Hou, Ruihao Zhu, and Abdallah El Ali. 2024. User experience design professionals’ perceptions of generative artificial intelligence. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–18.
- Robert J Sternberg, Jacques Lautrey, and Todd I Lubart. 2003. Where are we in the Field of Intelligence, How did we get here, and Where are we Going?American Psychological Association.
Cited by
(2000) · Time to knit · doi:10.1145/3772318.3808960
Human–AI Collaboration and Creative Behavior among Chinese College Students
Zhou, X. (2026) · International Conference on Educational and Information Technology · doi:10.1109/iceit68991.2026.11521236 · Cited by 2
Metadata
- Authors
- Fu, Y., Zhou, T., Han, B., Wang, M., Chen, Y., Gomes Da Costa Lai, Z., Pang, R. Y., Reinecke, K., Wobbrock, J. O., & Hiniker, A.
- Affiliations
- University of Washington (US)
- Human Centered Design and Engineering, University of Washington, Seattle, Washington, USA
- ORCID
- Venue
- Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2026-04-13
- Pages
- 1-17
- OpenAlex
- W7154165194
- Semantic Scholar
- f42eff11592488d36ba2685ccb764c54d76edebc
- Cited by
- 2
- References
- 80
- Citations by year
- 2026: 2
- FWCI
- 54.12
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- Topics
- Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Keywords
- generative AI, novice designers, creative thinking, AI-assisted design, design education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-27, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-27 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Decoupling of Usefulness and Novelty: Evaluating the Impact of Generative AI on Design Outputs and Novice Designers' Creative Thinking. https://app.cochat.ai/papers/10.1145/3772318.3791329
- The paper
- Fu, Y., Zhou, T., Han, B., Wang, M., Chen, Y., Gomes Da Costa Lai, Z., Pang, R. Y., Reinecke, K., Wobbrock, J. O., & Hiniker, A. (2026). Decoupling of Usefulness and Novelty: Evaluating the Impact of Generative AI on Design Outputs and Novice Designers' Creative Thinking. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3772318.3791329
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.