Understanding Nonlinear Collaboration between Human and AI Agents: A Co-design Framework for Creative Design
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
Creative design is a nonlinear process where designers generate diverse ideas in the pursuit of an open-ended goal and converge towards consensus through iterative remixing. In contrast, AI-powered design tools often employ a linear sequence of incremental and precise instructions to approximate design objectives. Such operations violate customary creative design practices and thus hinder AI agents’ ability to complete creative design tasks. To explore better human-AI co-design tools, we first summarize human designers’ practices through a formative study with 12 design experts. Taking graphic design as a representative scenario, we formulate a nonlinear human-AI co-design framework and develop a proof-of-concept prototype, OptiMuse. We evaluate OptiMuse and validate the nonlinear framework through a comparative study. We notice a subconscious change in people’s attitudes towards AI agents, shifting from perceiving them as mere executors to regarding them as opinionated colleagues. This shift effectively fostered the exploration and reflection processes of individual designers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Existing conceptualizations [56, 194] and frameworks [124, 200, 208] often lack granular detail on the specific operational control mechanisms needed to guide the design of practical systems.”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [208]
In the citing paper
“Agency distribution often involves a shared locus of control, potentially managed through dynamic allocation negotiated via interaction [47, 68], with varying granularity depending on the task [208].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [208]
In the citing paper
“The locus of control can be dynamic, shifting between human guidance and AI generation [8, 109], often operating at a fine granularity for detailed adjustments [208].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [208]
In the citing paper
“The granularity of control might be high-level (strategic direction [61]) or fine-grained (parameter tuning [208]) depending on the phase.”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [208]
Citations
Cites
Creativity syndrome: Integration, application, and innovation.
Mumford, M. D., & Gustafson, S. B. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.1.27 · Cited by 1,766 · cited 1×
“In the design process, creativity plays an irreplaceable role in promoting innovation [42, 64].”
BackgroundTotal Design: Integrated Methods for Successful Product Engineering
Pugh, S. (1991) · Medical Entomology and Zoology · Cited by 1,463
Managing the development of large software systems: concepts and techniques
Royce, W. W. (1987) · doi:10.5555/41765.41801 · Cited by 1,176
A cognitive process theory of writing
Flower, L., & Hayes, J. R. (1981) · College Composition & Communication · doi:10.58680/ccc198115885 · Cited by 1,025 · cited 2×
“The creative process is a purpose-driven journey in which individuals construct a hierarchical network of goals to serve as their guiding framework[15].”
BackgroundChoice overload: A conceptual review and meta-analysis
Chernev, A., Böckenholt, U., & Goodman, J. (2015) · Journal of Consumer Psychology · doi:10.1016/j.jcps.2014.08.002 · Cited by 858 · cited 1×
“To balance attention and choice overload [8], we limit to three selective options to highlight the features of multiple choices.”
BackgroundWizard of Oz Studies in HRI: A Systematic Review and New Reporting Guidelines
Riek, L. (2012) · Journal of Human-Robot Interaction · doi:10.5898/jhri.1.1.riek · Cited by 779 · cited 1×
“It is important to introduce supplementary constraints through scenarios in WoZ experiments [11, 17, 24, 54].”
BackgroundHoward, T. J., Culley, S. J., & Dekoninck, E. (2008) · Design Studies · doi:10.1016/j.destud.2008.01.001 · Cited by 731 · cited 8×
“However, they may suffer from poor user experience (e.g. low success rates [6]) due to the ignorance of natural collaboration process in creative design [28].”
BackgroundMethodInfluentialWizard of Oz studies — why and how
Dahlbäck, N., Jönsson, A., & Ahrenberg, L. (1993) · Knowledge-Based Systems · doi:10.1016/0950-7051(93)90017-n · Cited by 706
Fraser, N. M., & Gilbert, G. N. (1991) · Computer Speech & Language · doi:10.1016/0885-2308(91)90019-m · Cited by 442 · cited 2×
“It is important to introduce supplementary constraints through scenarios in WoZ experiments [11, 17, 24, 54].”
BackgroundMethodMapping the Landscape of Creativity Support Tools in HCI
Frich, J., MacDonald Vermeulen, L., Remy, C., Biskjaer, M. M., & Dalsgaard, P. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300619 · Cited by 410 · cited 1×
“In various related domains, such as requirement analysis [3, 13, 37, 55, 56] and psychology [15, 18, 52], research has focused on studying collaboration patterns among humans and highlighting the importance of requirement alignment and remixing.”
BackgroundLubart, T. (2005) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2005.04.002 · Cited by 365 · cited 4×
“Desired roles of AI in human-AI collaboration have been widely discussed from different perspectives (e.g. categories of human-computer interaction for creativity [36], tasks in ideation [38], computational creativity [44], work division in data storytelling [32, 33]).”
BackgroundInfluentialOh, C., Song, J., Choi, J., Kim, S., Lee, S., & Suh, B. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174223 · Cited by 301 · cited 2×
“The HCI community has investigated this issue based on the design lessons about developing creativity-support tools [2, 9, 31, 47, 61].”
BackgroundMethodWang, D., Churchill, E., Maes, P., Fan, X., Shneiderman, B., Shi, Y., & Wang, Q. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3381069 · Cited by 271
Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models
Louie, R., Coenen, A., Huang, C. Z., Terry, M., & Cai, C. J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376739 · Cited by 271 · cited 1×
“Moreover, by evaluating a set of AI-steering music design tools, Louie et al. [35] reported positive user feedback on human-AI collaboration.”
MethodCreative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257 · cited 3×
“I was kind of dizzy seeing the sentences became longer and longer...” Additionally, 𝑃 9 reported a lack of professional vocabulary, which affected users’ willingness to communicate about graphic design in the dialogue.”
BackgroundCommunity Support for Constructionist Learning
Bruckman, A. (1998) · Computer Supported Cooperative Work (CSCW) · doi:10.1023/a:1008684120893 · Cited by 251
Shi, Y., Gao, T., Jiao, X., & Cao, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610217 · Cited by 170
A literature review on individual creativity support systems
Wang, K., & Nickerson, J. V. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.04.035 · Cited by 166 · cited 6×
“Various HCI theories [26, 30, 36, 43, 44, 53, 61] have been conducted to provide guidelines for human-AI co-creativity.”
BackgroundMethodInfluentialSocial creativity: turning barriers into opportunities for collaborative design
Fischer, G. (2004) · Proceedings of the eighth conference on Participatory design Artful integration: interweaving media, materials and practices - PDC 04 · doi:10.1145/1011870.1011889 · Cited by 155 · cited 1×
“” The social nature of creativity, which highlights that much of our creative output is a result of interaction and communication with others [14], emphasizes the significance of collective creativity as a method that promotes innovation through collaborative efforts [29].”
Empirical research in requirements engineering: trends and opportunities
Ambreen, T., Ikram, N., Usman, M., & Niazi, M. (2016) · Requirements Engineering · doi:10.1007/s00766-016-0258-2 · Cited by 126 · cited 2×
“While RA in the design process has not been extensively studied, it has been a topic of longstanding discussion in other domains, particularly in software development [3, 13, 37, 55, 56].”
BackgroundApplying the Wizard-of-Oz framework to cooperative service discovery and configuration
Green, A., Hüttenrauch, H., & Eklundh, K. S. (2005) · IEEE RO-MAN · doi:10.1109/roman.2004.1374824 · Cited by 121 · cited 1×
“It is important to introduce supplementary constraints through scenarios in WoZ experiments [11, 17, 24, 54].”
BackgroundMünster, U., Müller-Wienbergen, F., Müller, O., University of Liechtenstein, Seidel, S., University of Liechtenstein, Becker, J., & University of Münster (2011) · Journal of the Association for Information Systems · doi:10.17705/1jais.00280 · Cited by 117 · cited 4×
“Various HCI theories [26, 30, 36, 43, 44, 53, 61] have been conducted to provide guidelines for human-AI co-creativity.”
BackgroundInfluentialVisualizing Biological Pathways: Requirements Analysis, Systems Evaluation and Research Agenda
Saraiya, P., North, C., & Duca, K. (2005) · Information Visualization · doi:10.1057/palgrave.ivs.9500102 · Cited by 106 · cited 2×
“While RA in the design process has not been extensively studied, it has been a topic of longstanding discussion in other domains, particularly in software development [3, 13, 37, 55, 56].”
BackgroundPatterns of Physical Design Remixing in Online Maker Communities
Oehlberg, L., Willett, W., & Mackay, W. E. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702175 · Cited by 101 · cited 3×
“Following the nonlinear framework, Opti-Muse enables parallel iteration [21] and remixing [7, 46, 58, 63].”
BackgroundMethodRemixing as a Pathway to Computational Thinking
Dasgupta, S., Hale, W., Monroy-Hernández, A., & Hill, B. M. (2016) · Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing · doi:10.1145/2818048.2819984 · Cited by 100 · cited 2×
“For further details, the relevant words of experts are provided in Appendix A. were fulfilled in a single iteration, the majority of designs (34 out of 36) underwent trial-and-error iterations.”
BackgroundFrom rookie to all-star: professional development in a graphic design social networking site
Marlow, J., & Dabbish, L. (2014) · Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing · doi:10.1145/2531602.2531651 · Cited by 88 · cited 1×
“Many design tools allow designers to merge their human collaborators’ ideas into their creative output iteratively [5, 12, 39].”
BackgroundAlthuizen, N., & Reichel, A. (2016) · Journal of Management Information Systems · doi:10.1080/07421222.2016.1172439 · Cited by 85 · cited 1×
“The HCI community has investigated this issue based on the design lessons about developing creativity-support tools [2, 9, 31, 47, 61].”
BackgroundRezwana, J., & Maher, M. L. (2022) · Creativity and Cognition · doi:10.1145/3527927.3532789 · Cited by 85 · cited 4×
“Similar to Rezwana’s finding in Penpal [53], we observe enhanced user engagement in OptiMuse, where AI was perceived as more collaborative than a mere tool.”
BackgroundMethodResultInfluentialHuman-AI Collaboration in a Cooperative Game Setting: Measuring Social Perception and Outcomes
Ashktorab, Z., Liao, Q. V., Dugan, C., Johnson, J., Pan, Q., Zhang, W., Kumaravel, S., & Campbell, M. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415167 · Cited by 79 · cited 3×
“However, as high-lighted by Ashktorab et al. [4], social perceptions exhibit significant differences when considering interaction with an AI compared to interacting with a human [4].”
BackgroundMethodInfluentialLi, H., Wang, Y., & Qu, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642726 · Cited by 77 · cited 1×
“Desired roles of AI in human-AI collaboration have been widely discussed from different perspectives (e.g. categories of human-computer interaction for creativity [36], tasks in ideation [38], computational creativity [44], work division in data storytelling [32, 33]).”
BackgroundSelf-citationNine Potential Pitfalls when Designing Human-AI Co-Creative SystemsPreprint
Buschek, D., Mecke, L., Lehmann, F., & Dang, H. (2021) · ERef Bayreuth (University of Bayreuth) · doi:10.48550/arxiv.2104.00358 · Cited by 73 · cited 2×
“However, they may suffer from poor user experience (e.g. low success rates [6]) due to the ignorance of natural collaboration process in creative design [28].”
BackgroundMethodReynante, B. M., Dow, S. P., & Mahyar, N. (2021) · Digital Government Research and Practice · doi:10.1145/3487607 · Cited by 61 · cited 1×
“In various related domains, such as requirement analysis [3, 13, 37, 55, 56] and psychology [15, 18, 52], research has focused on studying collaboration patterns among humans and highlighting the importance of requirement alignment and remixing.”
BackgroundProceedings of the Seventh International Conference on Computational Creativity
Llano, M. T., Christian Guckelsberger, B., Hepworth, R., Gow, J., Corneli, J., & Colton, S. (2016) · Cited by 46
Software requirement analysis: Research challenges and technical approaches
Demirel, S. T., & Daş, R. (2018) · International Symposium on Digital Forensic and Security (ISDFS) · doi:10.1109/isdfs.2018.8355322 · Cited by 35 · cited 2×
“While RA in the design process has not been extensively studied, it has been a topic of longstanding discussion in other domains, particularly in software development [3, 13, 37, 55, 56].”
BackgroundCheliotis, G., Hu, N., Yew, J., & Huang, J. (2014) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2531602.2531730 · Cited by 30 · cited 3×
“Following the nonlinear framework, Opti-Muse enables parallel iteration [21] and remixing [7, 46, 58, 63].”
BackgroundMethodCollaborative Creativity in TikTok Music Duets
O’Toole, K. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581380 · Cited by 28 · cited 1×
“For example, the TickRok music Duets enable a collective exploration of design space to increase ideation creativity [50].”
BackgroundLi, H., Wang, Y., Liao, Q. V., & Qu, H. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3552017 · Cited by 25
Self-citationA Word is Worth a Thousand Pictures: Prompts as AI Design Material
Kulkarni, C., Druga, S., Chang, M., Fiannaca, A., Cai, C. J., & Terry, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.12647 · Cited by 20 · cited 5×
“In OptiMuse, users’ textual descriptions allow rapid exploration of the design space [31].”
BackgroundResultInfluentialA Human-Computer Collaborative Editing Tool for Conceptual Diagrams
Pan, L., Yu, C., He, Z., & Shi, Y. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580676 · Cited by 20 · cited 2×
“When it comes to communication methods, both research [51, 57], de-sign practices, and our formative studies have observed designers employing various forms, including text, voice, gestures, sketches, graphical operations, and even eye movements, to convey and exchange design ideas and reflections.”
BackgroundMethodCollective Creativity through a Micro-Tasks Crowdsourcing Approach
Girotto, V. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818052.2874356 · Cited by 9 · cited 3×
“Such turn-based linear workflow hinders the natural nonlinear navigation among creative design stages [10, 19, 20, 27, 28, 45], which involves drawing inspiration from various parallel iterations and potentially modifying initial ideas along the way [21].”
BackgroundMethodTan, J. J., Qi, C., Gao, X., Lu, J., & Tan, Q. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.941448 · Cited by 8 · cited 4×
“For instance, through an empirical study on the Thingiverse (a 3D printing community), Tan et al. [58] revealed the contradictory impact of knowledge endowment heterogeneity on users’ quality and quantity of remixing.”
BackgroundMethodInfluentialInakage, M. (2007) · doi:10.1145/1306813.1306822 · Cited by 6 · cited 2×
“Navigating, summarizing, and discarding among the design choices are central to the nonlinear process of collective creativity [7, 29, 46, 58, 63] In alignment with the fourth and fifth design requirements addressing the cons of human-human collaboration, two required actions of AI agents are…”
BackgroundArtificial Intelligence in Music, Sound, Art and Design
(2022) · Lecture notes in computer science · doi:10.1007/978-3-031-03789-4 · Cited by 6
Li, H., Wang, Y. H., & Qu, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.15723 · Cited by 3 · cited 1×
“Desired roles of AI in human-AI collaboration have been widely discussed from different perspectives (e.g. categories of human-computer interaction for creativity [36], tasks in ideation [38], computational creativity [44], work division in data storytelling [32, 33]).”
BackgroundSelf-citation
Research works printed but not linked: 12
- 10.2307/356600 · doi:10.2307/356600 · Paper page
- John S Gero. 1990. Design prototypes: a knowledge representation schema for design. AI magazine 11, 4 (1990), 26–26.
- John S Gero and Udo Kannengiesser. 2004. The situated function–behaviour–structure framework. Design studies 25, 4 (2004), 373–391.
- Google. 2022. Imagen: Text-to-Image Diffusion Models. https://imagen.research.google/. Last accessed on 14.09.2023.
- Google. 2022. Parti: Pathways Autoregressive Text-to-Image Model. https://parti.research.google/. Last accessed on 14.09.2023.
- Matthew Guzdial and Mark Riedl. 2019. An interaction framework for studying co-creative ai. (2019). https://doi.org/10.48550/arXiv:1903.09709 arXiv:arXiv:1903.09709 · arXiv · doi:10.48550/arxiv:1903.09709
- Armand Hatchuel and Benoit Weil. 2003. A new approach of innovative Design: an introduction to CK theory.. In DS 31: Proceedings of ICED 03, the 14th International Conference on Engineering Design. the Design Society, Stockholm, Sweden, 109–110.
- Martin Maguire and Nigel Bevan. 2002. User requirements analysis: a review of supporting methods. In IFIP World Computer Congress, TC 13. Springer, Boston, MA, 133–148.
- Mary Lou Maher. 2012. Computational and collective creativity: Who’s being creative?. In Proceedings of the International Conference on Computational Creativity. Association for Computational Creativity, Jönköping, Sweden, 67–71.
- SANTIAGO NEGRETE YANKELEVICH and NORA ANGELICA MORALES ZARAGOZA. 2014. The apprentice framework: planning and assessing creativity. In Proceedings of the International Conference on Computational Creativity. Association for Computational Creativity, Jönköping, Sweden, 280–283.
- Feng Yue, Ma Qiao-ge, and Ren Nan. 2019. Improving makers’ remix contribution behavior in the open innovation communities——An example of Thingiverse. In 2019 4th International Conference on Social Sciences and Economic Development (ICSSED 2019). Atlantis Press, GX Dordrecht, The Netherlands, 224–229.
- Shahid Yusuf. 2009. From creativity to innovation. Technology in society 31, 1 (2009), 1–8.
Cited by
AIdeation: Designing a Human-AI Collaborative Ideation System for Concept Designers
Wang, W.-F., Lu, C.-T., Ponsa i Campanyà, N., Chen, B.-Y., & Chen, M. Y. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714148 · Cited by 50
Zhang, S., Wang, H., & Yi, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757594 · Cited by 36
“Agency distribution often involves a shared locus of control, potentially managed through dynamic allocation negotiated via interaction [47, 68], with varying granularity depending on the task [208].”
InfluentialTang, Y., Zhang, N., Ciancia, M., & Wang, Z. (2024) · Companion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing · doi:10.1145/3678884.3681890 · Cited by 31
“Many artists and researchers are further exploring the support that AI-generated image tools provide for artistic creation [28, 46, 48, 49] and creativity [29, 30, 58].”
Designing Co-Creative Systems: Five Paradoxes in Human–AI Collaboration
Salma, Z., Hijón‐Neira, R., & Pizarro, C. (2025) · Information · doi:10.3390/info16100909 · Cited by 48
“The description of the executor position [18] brings up a significant point: AI frequently generates results that are not explicable.”
InfluentialMaking the case for introducing generative artificial intelligence (AI) into design curricula
Fleischmann, K. (2024) · Art Design & Communication in Higher Education · doi:10.1386/adch_00088_1 · Cited by 45
Metadata
- Authors
- Zhou, J., Li, R., Tang, J., Tang, T., Li, H., Cui, W., & Wu, Y.
- Affiliations
- Zhejiang University (CN)
- State Key Lab of CAD&CG, Zhejiang University, China
- Hong Kong University of Science and Technology (HK)
- Microsoft Research Asia (China) (CN)
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-16
- OpenAlex
- W4396832811
- Semantic Scholar
- feacb3129097876863b3e25f5d750454a58e73b7
- Cited by
- 107
- References
- 43
- Citations by year
- 2026: 51 · 2025: 46 · 2024: 10
- FWCI
- 137.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62302440, U22A2032)
- Ministry of Education, India
- Topics
- Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- human-AI collaboration, creative design, AI agents, co-design framework, graphic design, designer practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.