Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models
Open accessThe DOI matches its Crossref record; 36 of the 37 references it cites were found in a registry or an index.
Abstract
While generative deep neural networks (DNNs) have demonstrated their capacity for creating novel musical compositions, less attention has been paid to the challenges and potential of co-creating with these musical AIs, especially for novices. In a needfinding study with a widely used, interactive musical AI, we found that the AI can overwhelm users with the amount of musical content it generates, and frustrate them with its non-deterministic output. To better match co-creation needs, we developed AI-steering tools, consisting of Voice Lanes that restrict content generation to particular voices; Example-Based Sliders to control the similarity of generated content to an existing example; Semantic Sliders to nudge music generation in high-level directions (happy/sad, conventional/surprising); and Multiple Alternatives of generated content to audition and choose from. In a summative study (N=21), we discovered the tools not only increased users' trust, control, comprehension, and sense of collaboration with the AI, but also contributed to a greater sense of self-efficacy and ownership of the composition relative to the AI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Creative industries employ vision-aligned proactive and semiproactive agency.”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [109]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Mapping interaction flows reveals prevalent patterns and underexplored areas, such as the sparse use of 'Auditory Interaction' [84, 109, 164], signaling innovation opportunities in domains like assistive technology or music co-creation.”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [109]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AI agency patterns are typically co-operative or semi-active [66, 191], often involving a shared or dynamically negotiated locus of control [8, 47, 109].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [109]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The locus of control can be dynamic, shifting between human guidance and AI generation [8, 109], often operating at a fine granularity for detailed adjustments [208].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [109]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In contrast to software engineering, other domains including Human-Computer Interaction (HCI), organization and business management, and cognitive science have explored the impact of LLMs on creativity.”
From The Impact of Generative AI on Creativity in Software Development: A Research Agenda (2025) · cites this paper as [77]
Experimental — accuracy not yet measured
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 1×
“To analyze the quantitative measures, we conducted paired t-tests using Benjamani-Hochberg correction [3] to account for the 15 planned-comparisons (using a false discovery rate Q = 0 .”
MethodDevelopment of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · cited 2×
“Effort: Users answered the effort question of the NASA-TLX [25], where 1=very low and 7=very high.”
Method(2016) · Cited by 11,679
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · cited 1×
“Trust : Participants rated the system along Mayer’s dimensions of trust [35]: capability, benevolence, and integrity.”
BackgroundGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 3×
“Previous research suggests benefits of educating users about the AI and its capabilities [1], or providing onboarding materials and exercises [6].”
BackgroundCai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594 · cited 1×
“Previous research suggests benefits of educating users about the AI and its capabilities [1], or providing onboarding materials and exercises [6].”
BackgroundSelf-citationBoulanger-Lewandowski, N., Bengio, Y., & Vincent, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6392 · Cited by 459 · cited 1×
“Research has demonstrated the potential for modeling and synthesizing music, ranging from singlevoice sequences [13] and multi-part music [19, 34], to music with variable parts at each time step [4] and music with longterm structure over minutes [30, 37, 26].”
BackgroundDeepBach: a Steerable Model for Bach Chorales GenerationPreprint
Hadjeres, G., Pachet, F., & Nielsen, F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.01010 · Cited by 443 · cited 7×
“The conventional interface does include interactive control features via the infillmask feature (present in both conditions) which enables users to crop any region of music to be regenerated [11, 24].”
BackgroundMethodInfluentialOh, C., Song, J., Choi, J., Kim, S., Lee, S., & Suh, B. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174223 · Cited by 301
Research works printed but not linked: 6
- 8. Kate Compton and Michael Mateas. 2015. Casual Creators. In Proceedings of the Sixth International Conference on Computational Creativity (ICCC 2015),
- 9. Hannu Toivonen, Simon Colton, Michael Cook, and Dan Ventura (Eds.). Brigham Young University, Park City, Utah, 228-235. http://computationalcreativity.net/ iccc2015/proceedings/10_2Compton.pdf
- 12. Monica Dinculescu and Cheng-Zhi Anna Huang. 2019. Coucou: An Expanded Interface for Interactive Composition with Coconet, through Flexible Inpainting. (2019). https://coconet.glitch.me/
- 33. Hannu Toivonen, Simon Colton, Michael Cook, and Dan Ventura (Eds.). Brigham Young University, Park City, Utah, 236-243. http://computationalcreativity.net/ iccc2015/proceedings/10_3Jacob.pdf
- 36. Feynman Liang. 2016. BachBot: Automatic Composition in the Style of Bach Chorales. Masters thesis, University of Cambridge (2016).
- 42. Ralf Schwarzer and Matthias Jerusalem. 1995. Generalized Self-efficacy Scale. Measures in Health Psychology: A User's Portfolio. Causal and Control Beliefs 1, 1 (1995), 35-37.
Cited by
Design Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501825 · Cited by 639
“[30] introduced CoCoCo, an AI music creation system which lets users move sliders to tune their generated music to be "happier" or "sadder".”
BackgroundInfluentialDesign Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2021) · arXiv · doi:10.48550/arxiv.2109.06977 · Cited by 635
“[30] introduced CoCoCo, an AI music creation system which lets users move sliders to tune their generated music to be "happier" or "sadder".”
BackgroundInfluentialWu, T., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517582 · Cited by 483
“Contrary to recent work in human-AI interaction, which primarily examined how to increase AI controllability through exposing knobs within a model [44, 49], our work opens up the possibility of steering AI using the model itself as units to control.”
BackgroundSelf-citationWordcraft: Story Writing With Large Language Models
Yuan, A., Coenen, A., Reif, E., & Ippolito, D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511105 · Cited by 374
“There have been many studies of humanAI co-creation across domains, including drawing [8, 27, 31], music [15, 22, 25], video game content creation [24], and design ideation [16].”
BackgroundSelf-citationRezwana, J., & Maher, M. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3519026 · Cited by 342
“AI ability alone does not ensure a positive collaborative experience of users with the AI [100] and interaction is more critical than algorithms where interaction with the users is essential [152].”
BackgroundHelp Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
Kim, S. S. Y., Watkins, E. A., Russakovsky, O., Fong, R., & Monroy-Hernández, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581001 · Cited by 233
“…agents [8, 25, 31, 92], embedded systems [4, 23, 38, 40, 53, 57, 62, 63, 89, 91, 128]) in different task contexts (e.g., content generation [65, 70, 142], medical diagnosis [23, 40, 128], content moderation [53, 62], deception detection [63, 89], cooperative games [8], and fine-grained visual…”
BackgroundDesign Principles for Generative AI Applications
Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642466 · Cited by 232
“…classifcation 9 Some user interfaces to generative AI systems allow users to specify their intent via sketches and gestures [32], UI controls [103, 105], video-scanned body movements [174], and even various forms of improv performance [15, 68]. use cases in which false positives and false…”
BackgroundInvestigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219
“Recent HCI works also explored the approach of “explainability through interaction" for generative models, by allowing users to interact with the input or guiding output generation process [59, 60, 86, 103].”
BackgroundDirectGPT: A Direct Manipulation Interface to Interact with Large Language Models
Masson, D., Malacria, S., Casiez, G., & Vogel, D. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642462 · Cited by 107
“For example, using sliders to manipulate continuous parameters that impact the generation of images [16, 56] and music [35].”
Ji, S., Yang, X., & Luo, J. (2023) · ACM Computing Surveys · doi:10.1145/3597493 · Cited by 220
Human-AI interaction research agenda: A user-centered perspective
Tingting, J., Sun, Z., Fu, S., & Lv, Y. (2024) · Data and Information Management · doi:10.1016/j.dim.2024.100078 · Cited by 217
“How knowledge workers think generative AI will (not) transform their industries,”
Woodruff, A., Shelby, R., Kelley, P. G., Rousso-Schindler, S., Smith-Loud, J., & Wilcox, L. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642700 · Cited by 105
“While considerable work within the HCI community has explored how human and AI systems can collaborate for higher quality and more creative outcomes [34, 76, 99, 146, 153], this question encourages research into how these collaborations can be designed to support knowledge workers [63], as well as the design of system…”
BackgroundJi, S., Luo, J., & Yang, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06801 · Cited by 203
“[288] developed Cococo (collaborative co creation) based on Coconet, a music editor web-interface for novice-AI co-creation.”
BackgroundLarge-scale Text-to-Image Generation Models for Visual Artists’ Creative Works
Ko, H.-K., Park, G., Jeon, H., Jo, J., Kim, J., & Seo, J. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584078 · Cited by 189
“Task Composition Human-AI music co-creation [57]; Music for video generation [23]; Drum beat creation [93] 9 12.”
BackgroundYang, D., Zhou, Y., Zhang, Z., Li, T. J.-J., & Lc, R. (2022) · CityUHK Scholars · Cited by 185
Han, Y., Qiu, Z., Cheng, J., & LC, R. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642133 · Cited by 104
“Thetasksincludecomprehension andcreativewriting[7,21,42,62],mixed-initiativestorytelling [58],artisticvisualcreation[5,8,25,47],audioproduction[37,48, 56],videogeneration[20],etc. Onestudyexploredinvolvingmore thanoneGenAIagentinusingChatGPTtogenerateastoryand visualizeitwithStableDiffusion[22].”
Determinants of Students’ Satisfaction with AI Tools in Education: A PLS-SEM-ANN Approach
Almufarreh, A. (2024) · Sustainability · doi:10.3390/su16135354 · Cited by 165
Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
Yildirim, N., Pushkarna, M., Goyal, N., Wattenberg, M., & Viégas, F. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580900 · Cited by 104
“icant amount of research investigated the challenges of designing human-AI interaction, such as intelligibility [1], explainability [60, 98], user control [62, 87], feedback [89], error recovery [41], and setting user expectations [47, 53].”
BackgroundCivit, M., Civit-Masot, J., Cuadrado, F., & Escalona, M. J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118190 · Cited by 158
A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning
Kaluarachchi, T., Reis, A., & Nanayakkara, S. (2021) · Sensors · doi:10.3390/s21072514 · Cited by 124
“[85] investigated how to adjust an AI music generation tool to minimize user burden with user experiments.”
MethodIs It AI or Is It Me? Understanding Users’ Prompt Journey with Text-to-Image Generative AI Tools
Mahdavi Goloujeh, A., Sullivan, A., & Magerko, B. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642861 · Cited by 89
“Challenges of content generation with AI include the open-ended nature of generative outputs, lack of user agency, misalignment with users’ creative goals, and information overload [10, 35, 48].”
PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions
Chung, J. J. Y., & Adar, E. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606777 · Cited by 96
“sad face) to sliders on a fixed continuous semantic scale [15, 22, 55, 61] (e.”
Suh, S., Chen, M., Min, B., Li, T. J.-J., & Xia, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642400 · Cited by 148
Chang, M., Druga, S., Fiannaca, A. J., Vergani, P., Kulkarni, C., Cai, C. J., & Terry, M. (2023) · Creativity and Cognition · doi:10.1145/3591196.3593515 · Cited by 108
“AI-based CST systems have been produced to support artistic generation in domains such as fashion and product design [34, 50, 56], music creation [32, 38, 41], drawing [16, 17, 36, 46], visual design and story-boarding [57, 61], and storytelling [31, 49].”
BackgroundSelf-citationLAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing
Wang, B., Li, Y., Lv, Z., Xia, H., Xu, Y., & Sodhi, R. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645143 · Cited by 63
“This includes areas such as story writing [15, 20, 50, 82], music composition [26, 46, 47], comic creation [64], and game design [85].”
BackgroundAI as Social Glue: Uncovering the Roles of Deep Generative AI during Social Music Composition
Suh, M. (., Youngblom, E., Terry, M., & Cai, C. J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445219 · Cited by 138
“Human-AI co-creation has recently been investigated in domains such as drawing [18, 20, 42, 43, 59], music composition [54], creative writing [26, 37], design ideation [46, 51, 69], visual metaphors [11], video game content generation [28, 47], improvisation [38], public displays [52], and dance [53].”
BackgroundMethodInfluentialSelf-citationGmeiner, F., Yang, H., Yao, L., Holstein, K., & Martelaro, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580999 · Cited by 130
“Previousworkonnovice-AI music co-creation has also found that users systematically tested AI limitations to hone their mental model of the system’s behavior [48].”
BackgroundPerfection Not Required? Human-AI Partnerships in Code Translation
Weisz, J. D., Muller, M., Houde, S., Richards, J., Ross, S. I., Martinez, F., Agarwal, M., & Talamadupula, K. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450656 · Cited by 129
“For example, they have been used in creative domains to create or alter media such as images, videos, and artwork [10, 46, 54, 56, 92], music [11, 45, 57], and writing [21, 32, 76].”
BackgroundInfluentialPersonas for Artificial Intelligence (AI) an Open Source Toolbox
Holzinger, A., Kargl, M., Kipperer, B., Regitnig, P., Plass, M., & Muller, H. (2022) · IEEE Access · doi:10.1109/access.2022.3154776 · Cited by 96
“AI systems can generate output that is non-deterministic and unexpected, e.g. co-creating musical content [17].”
BackgroundWang, A. Y., Wang, D., Drozdal, J., Muller, M., Park, S., Weisz, J. D., Liu, X., Wu, L., & Dugan, C. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3489465 · Cited by 81
“Inspired by prior studies that implement AI systems to work together with human [36, 69], we believe the system will work better if it has both the automated documentation capability and the capability that allows users to directly manipulate the documentation.”
MethodRePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions
Wang, Y., Shen, S., & Lim, B. Y. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581402 · Cited by 120
“In recent works, XAI has been used to support computer-aided translation [17], music creation [45], AI-directed crowd ideation [83], and concise feedback generation [82].”
MethodInvestigating Code Generation Performance of ChatGPT with Crowdsourcing Social Data
Feng, Y., Vanam, S., Cherukupally, M., Zheng, W., Qiu, M., & Chen, H. (2023) · 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC) · doi:10.1109/compsac57700.2023.00117 · Cited by 117
“…the advancements in large language models (LLMs) and generative models have revolutionized many applications, including free text generation, question answering, and document summarization, enabling a wide range of real-world services such as AI robot lawyers [1] and AI music co-creation [2].”
The Intersection of Users, Roles, Interactions, and Technologies in Creativity Support Tools
Chung, J. J. Y., He, S., & Adar, E. (2021) · Designing Interactive Systems Conference 2021 · doi:10.1145/3461778.3462050 · Cited by 83
“Art-making CST innovations have enabled new forms of interaction for more diverse users, from novices [73, 89] and experts [46, 74] to those with disabilities [97].”
BackgroundInfluentialBiermann, O. C., Ma, N. F., & Yoon, D. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533506 · Cited by 116
“In [60], AI music co-creation is supported via steering tools which enable composition.”
BackgroundHeigl, R. M. (2025) · Management Review Quarterly · doi:10.1007/s11301-025-00494-9 · Cited by 122
“Factors such as gender (Heigl et al. 2023), personal beliefs about GenAI (Hong and Curran 2019; Mateja et al. 2024), art expertise (Louie et al. 2020; Mateja et al. 2024; Samo and Highhouse 2023), GenAI literacy (Wang et al. 2024), and overall competence (Lattika et al. 2023) are presented as…”
InfluentialPromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
Wang, Z., Huang, Y., Song, D., Ma, L., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642803 · Cited by 115
“These results indicate that the effectiveness of PromptCharm is not strongly correlated to user expertise.”
"Explorers of Unknown Planets": Practices and Politics of Artificial Intelligence in Visual Arts
Caramiaux, B., & Fdili Alaoui, S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555578 · Cited by 114
“Among the existing research, we can cite applications in game design [34], fashion design [40], moodboarding [44], abstract drawing [16], or music [30, 49].”
BackgroundLiu, H., Lai, V., & Tan, C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479552 · Cited by 111
“AI assistance has been shown to improve people’s efficiency in tasks such as transcription by enhancing their computational capacity [16, 35], support creativity in producing music [15, 41, 45], and even allow the visually impaired to “see” images [22, 68].”
BackgroundReelFramer: Human-AI Co-Creation for News-to-Video Translation
Wang, S., Menon, S., Long, T., Henderson, K., Li, D., Crowston, K., Hansen, M., Nickerson, J. V., & Chilton, L. B. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642868 · Cited by 55
“Generative AI has already been hugely successful in supporting creative tasks, including music composition [24], visual design [22, 44], and writing [7, 11, 39].”
BackgroundThe impact of generative AI on school music education: Challenges and recommendations
Cheng, L. (2025) · Arts Education Policy Review · doi:10.1080/10632913.2025.2451373 · Cited by 113
“Scholarly works often use the term ‘co-creation’ to describe the use of these generative tools to create musical content (e.g., Feuerriegel et al., 2024; Louie et al., 2020; Shu et al., 2021), highlighting the creative inputs from both users and generative AI.”
VRCopilot: Authoring 3D Layouts with Generative AI Models in VR
Zhang, L., Pan, J., Gettig, J., Oney, S., & Guo, A. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676451 · Cited by 109
“TheaccelerationofAIcapabilitieshas enabledhuman-AIco-creation in domains such as drawing [15, 19], creative writing [13], video game content creation [25], and music composition [28, 37].”
Zhou, J., Li, R., Tang, J., Tang, T., Li, H., Cui, W., & Wu, Y. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642812 · Cited by 107
“Moreover, by evaluating a set of AI-steering music design tools, Louie et al. [35] reported positive user feedback on human-AI collaboration.”
MethodBridging the Gap Between UX Practitioners’ Work Practices and AI-Enabled Design Support Tools
Lu, Y., Zhang, C., Zhang, I., & Li, T. J.-J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519809 · Cited by 63
“With the advancement of ML, many researchers have explored human-AI co-creation in creative domains including writing [4, 12, 22], music [29], drawing [7, 15, 20, 37], etc.”
BackgroundID.8: Co-Creating Visual Stories with Generative AI
Antony, V. N., & Huang, C.-M. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3672277 · Cited by 106
“Providing semantically equivalent proxies for advanced model parameters in form of sliders or gestural inputs may also be a viable avenue to democratize advanced model control parameters [15, 37].”
BackgroundSelf-citationToo Late to be Creative? AI-Empowered Tools in Creative Processes
Hwang, A. H.-C. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503549 · Cited by 103
“Some common internal struggles when collaborating with AI include the above-mentioned issue of trust, as well as the perceived self-efcacy in teams, [3, 24, 46].”
BackgroundGenAICHI: Generative AI and HCI
Muller, M., Chilton, L. B., Kantosalo, A., Martin, C. P., & Walsh, G. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503719 · Cited by 101
“Systems that are capable of producing novel instances are only recently documented [e.g., 3, 12, 13], and tooling that allows domain experts to control those novel or creative outcomes is even rarer [e.g., 27].”
BackgroundAngleKindling: Supporting Journalistic Angle Ideation with Large Language Models
Petridis, S., Diakopoulos, N., Crowston, K., Hansen, M., Henderson, K., Jastrzebski, S., Nickerson, J. V., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580907 · Cited by 98
“Generative models are being successfully applied to support a number of creative tasks, including music composition [38], designing visual art [36] [35], and writing [24] [14] [47].”
MethodDeepThInk: Designing and probing human-AI co-creation in digital art therapy
Du, X., An, P., Leung, J., Li, A., Chapman, L. E., & Zhao, J. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103139 · Cited by 95
Yue, B., & Li, H. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1277861 · Cited by 94
AI Song Contest: Human-AI Co-Creation in Songwriting
Huang, C.-Z. A., Koops, H. V., Newton-Rex, E., Dinculescu, M., & Cai, C. J. (2020) · arXiv · doi:10.48550/arxiv.2010.05388 · Cited by 90
“Even when composing short, two-bar counterpoints, it can be challenging for novice musicians to partner with a deep generative model: users desire greater agency, control, and sense of author-ship vis-a-vis the AI during co-creation [45].”
BackgroundMethodInfluentialSelf-citationRapp, A., Lodovico, C. D., Torrielli, F., & Di, L. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103375 · Cited by 89
“A first line of studies builds on previous research on AI-based creativity support tools and human-AI co-creation (Chang et al., 2023; Louie et al., 2020; Davis et al., 2016; Hodhod and Magerko, 2016).”
Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions
Vasconcelos, H., Bansal, G., Fourney, A., Liao, Q. V., & Vaughan, J. W. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3702320 · Cited by 90
Wan, Q., Hu, S., Zhang, Y., Wang, P., Wen, B., & Lu, Z. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637361 · Cited by 88
“Along this line of research, recent studies have extensively explored the potential of AI, notably generative models, for creative tasks such as drawing [18, 19, 41, 52, 77], contemporary art composition [10], fashion design [35], musical composition [46], digital mood board creation [37], and so on.”
BackgroundInvestigating and Designing for Trust in AI-powered Code Generation Tools
Wang, R., Cheng, R., Ford, D., & Zimmermann, T. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658984 · Cited by 87
“This also echoes prior research in the context of AI-powered music generation which indicated that enabling users to steer AI behavior increases trust in AI [39].”
BackgroundResultHe, Z., Su, J., Chen, L., Wang, T., & Lc, R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711006 · Cited by 88
“Louie et al. applied these technologies to empower untrained users to engage in complex creative endeavors such as music composition, traditionally requiring specific skills and training [43].”
PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation
Liu, Y., Sharma, P., Oswal, M., Xia, H., & Huang, Y. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735789 · Cited by 86
“By allowing fine-grained control over persona attributes, systems can encourage diverse and adaptable exploration of multiple perspectives and critical thinking.”
Rezwana, J., & Maher, M. L. (2022) · Creativity and Cognition · doi:10.1145/3527927.3532789 · Cited by 85
“AI ability alone does not ensure a positive collaborative experience of users with the AI [28] and interaction is more critical than algorithms in the systems where interaction with the users is essential [40].”
BackgroundC2Ideas: Supporting Creative Interior Color Design Ideation with a Large Language Model
Hou, Y., Yang, M., Cui, H., Wang, L., Xu, J., & Zeng, W. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642224 · Cited by 81
“To overcome LLMs’ performance limitations in specifc domains, we encode multiple domain-specifc knowledge elements related to interior color design into prompts integrated within our framework.”
Jiang, E., Toh, E., Molina, A., Olson, K., Kayacik, C., Donsbach, A., Cai, C. J., & Terry, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501870 · Cited by 79
“For example, Cococo [33] integrates a generative music model into an end-user application.”
BackgroundSelf-citationZheng, G., & Brintrup, A. M. (2025) · International Journal of Production Research · doi:10.1080/00207543.2025.2543964 · Cited by 81
Osone, H., Lu, J.-L., & Ochiai, Y. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3450391 · Cited by 74
“Substantial works of AI-powered tools have focused on devising interfaces that can not only be useful for user co-creation but also be coordinated with AI’s intelligent modules, such as machine learning, for creative composition [15, 17–19, 28, 40] such as drawing, design, and music.”
BackgroundA Word is Worth a Thousand Pictures: Prompts as AI Design Material
Kulkarni, C., Druga, S., Chang, M., Fiannaca, A., Cai, C. J., & Terry, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.12647 · Cited by 20
Self-citationFlick, C., & Worrall, K. (2022) · Springer series on cultural computing · doi:10.1007/978-3-031-10960-7_5 · Cited by 73
Huang, R., Lin, H., Chen, C., Zhang, K., & Zeng, W. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642824 · Cited by 72
“The average age of the participants is 24.9 (2.22% in [0,18], 53.33% in (18,25], 35.56% in (26,30], 8.89% in (31,40]).”
BackgroundInfluentialLaMPost: Design and Evaluation of an AI-assisted Email Writing Prototype for Adults with Dyslexia
Goodman, S. M., Buehler, E., Clary, P., Coenen, A., Donsbach, A., Horne, T. N., Lahav, M., MacDonald, R., Michaels, R. B., Narayanan, A., et al. (2022) · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3517428.3544819 · Cited by 71
“, [38]), and their overall satisfaction with their emails.”
BackgroundSelf-citationDeep learning’s shallow gains: a comparative evaluation of algorithms for automatic music generation
Yin, Z., Reuben, F., Stepney, S., & Collins, T. (2023) · Machine Learning · doi:10.1007/s10994-023-06309-w · Cited by 68
“While it is possible to achieve ostensibly impressive results with nov-ice users or listeners (Louie et al., 2020), it does not benefit the broader purpose of measuring true progress (or lack thereof) in the field.”
BackgroundJin, Y., Yoon, J., Self, J. A., & Lee, K. (2024) · Archives of Design Research · doi:10.15187/adr.2024.07.37.3.25 · Cited by 66
Controlling AI Agent Participation in Group Conversations: A Human-Centered Approach
Houde, S., Brimijoin, K., Muller, M. J., Ross, S., Moran, D., Gonzalez, G. E., Kunde, S., Foreman, M. A., & Weisz, J. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712089 · Cited by 68
“…with generative AI systems has explored ways to give users various controls over the outputs of the underlying generative model (e.g. [8, 10, 39, 40]), but we recognized the types of controls users needed were not solely related to the content of Koala’s messages, but also to the ways those…”
Decision Making Strategies and Team Efficacy in Human-AI Teams
Munyaka, I., Ashktorab, Z., Dugan, C., Johnson, J., & Pan, Q. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579476 · Cited by 42
“…AI systems assist in decision-making in a variety of areas such as medical support systems [17], pedagogical tools [31, 32], and creative tools [43, 46], investigating how users navigate and understand the Human-AI relationship in more complex teams beyond two parties (i.e., human-human-AI or…”
BackgroundPromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers’ Workflows
Petridis, S., Terry, M. A., & Cai, C. J. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661613 · Cited by 63
“These models are being used to create prototypes that support a variety of tasks, including music composition [24], designing visual art [22, 23, 38], simulating social communities [27, 28] and writing [4, 7, 16, 29, 48].”
MethodSelf-citationA Survey of AI Music Generation Tools and ModelsPreprint
Zhu, Y., Baca, J., Rekabdar, B., & Rawassizadeh, R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.12982 · Cited by 63
“Artificial intelligence (AI) music generation tools provide musicians and composers with new and innovative ways to create music, which has not only facilitated the expression of users’ musical intent but also had a significant impact on their creative ownership and confidence in their ability to collaborate with AI t…”
BackgroundBelghith, Y., Goloujeh, A. M., Magerko, B. S., Long, D., McKlin, T., & Roberts, J. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642332 · Cited by 62
“[61].”
BackgroundLiu, Y., Chen, S., Cheng, H., Yu, M., Ran, X., Mo, A., Tang, Y., & Huang, Y. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642698 · Cited by 61
“Many human-AI co-creation systems utilize generative AI technologies [45, 68].”
Backgroundvan Berkel, N., Sarsenbayeva, Z., & Gonçalves, J. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102954 · Cited by 61
Zhao, Y., Yang, M., Lin, Y., Zhang, X., Shi, F., Wang, Z., Ding, J., & Ning, H. (2025) · Electronics · doi:10.3390/electronics14061197 · Cited by 63
Louie, R., Engel, J., & Huang, C.-Z. A. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511159 · Cited by 30
“When designing a steering interface for our experiments, we drew upon HCI research about building better interactive tools that provide users (e.g., novice composers) more control and agency when creating with deep generative models [37].”
BackgroundMethodInfluentialSelf-citationAn Empirical Study on How People Perceive AI-generated Music
Chu, H., Kim, J., Kim, S., Lim, H., Lee, H., Jin, S., Lee, J., Kim, T.-H., & Ko, S. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557235 · Cited by 62
“AI-based music generation can be an alternative because it makes the music composition process much easier and more convenient for both professional and novice content creators [40].”
BackgroundInfluentialDesigning Interactions with Generative AI for Art and Creativity: A Systematic Review and Taxonomy
Hu, X., Xing, Y., Cai, X., Zhao, Y., Cook, M., Borgo, R., & Neate, T. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735843 · Cited by 61
“Metamorpheus[170],GenQuery[143], AIStory[69] Timeline/Storyline/Step Sequencer Cococo [97] Interpolating Slider/Region/ Graph PromptPaint [38], Mixplorer [85] Simultaneous Structured Divergence/Convergence Systems provide structured interfaces that guide users in exploring diverse ideas…”
The Role of Artificial Intelligence in Enhancing Design Innovation and Sustainability
Agboola, O. P. (2024) · Smart Design Policies · doi:10.38027/smart-v1n1-2 · Cited by 59
Enhancing Data Literacy On-Demand: LLMs as Guides for Novices in Chart Interpretation
Choe, K., Lee, C., Lee, S., Song, J., Cho, A., Kim, N. W., & Seo, J. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3413195 · Cited by 58
“Expectations of AI systems vary, ranging from seeking help with mundane tasks [34], [36], desiring a partnership with a creative entity [31], [32], to using AI as a source of entertainment during tasks [34].”
Patchview: LLM-powered Worldbuilding with Generative Dust and Magnet Visualization
Chung, J. J. Y., & Kreminski, M. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676352 · Cited by 57
“Some tools allow steering or evaluation of multiple generation results on dimensional spaces of attributes, represented in either sliders [38], mixed color spaces [15], temporal line drawing [17], or scatter plots organized in grids [50].”
BackgroundKim, T. S., Lee, Y., Chang, M., & Kim, J. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606833 · Cited by 56
“Users may then need to repeatedly experiment with generation configurations to understand their effect [15, 61, 87]—expending significant effort.”
BackgroundMethodInfluentialWorkplace security and privacy implications in the GenAI age: A survey
Diro, A. A., Kaisar, S., Saini, A., Fatima, S., Pham, H., & Erba, F. (2025) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2024.103960 · Cited by 57
Toward General Design Principles for Generative AI Applications
Weisz, J. D., Muller, M., He, J., & Houde, S. (2023) · arXiv · doi:10.48550/arxiv.2301.05578 · Cited by 54
“Providing users with controls in generative applications can improve their experience by increasing their efficiency, comprehension, and ownership of generated outcomes [34].”
BackgroundMethodInfluentialKeyframer: Empowering Animation Design using Large Language ModelsPreprint
Tseng, T., Cheng, R., & Nichols, J. W. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.06071 · Cited by 55
“…to entry for engaging in creative practice, while the interface paradigms of co-creating with AI are being proposed in domains such as graphic design [12, 25, 36, 44], software development [6, 49], creative computing [3, 32], UI design [43], writing [26, 46], 3D CAD [27, 37], and music [38].”
BackgroundXu, X. (., Konnova, A., Gao, B., Peng, C., Vo, D., & Dow, S. P. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713649 · Cited by 53
“Multi-modal AI models can support even more advanced creative tasks like music composition [39, 54] and video production [13].”
Yang, T., & Nazir, S. (2022) · Soft Computing · doi:10.1007/s00500-022-06734-4 · Cited by 50
“Louie et al. (2020) have suggested artificial intelligence grounded intelligent system which allows the users to generate the music according to their comfort.”
BackgroundRetractedNguyen, H., & Nguyen, A. (2024) · Journal of Science Education and Technology · doi:10.1007/s10956-024-10175-z · Cited by 49
“In those tasks, the AI tools take on assistive roles rather than making decisions for the users (Louie et al., 2020).”
GANCollage: A GAN-Driven Digital Mood Board to Facilitate Ideation in Creativity Support
Wan, Q., & Lu, Z. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3596072 · Cited by 46
“Although in the feld of computer-supported creativity [8, 19, 28], AI and other computing technologies have been deployed in many meaningful ways, little attention yet has been paid to how explainability of these AI models should be communicated.”
BackgroundThe Co-Creative Design Framework for Hybrid Intelligence
Davis, N., Sherson, J. F., & Rafner, J. F. (2025) · Creativity and Cognition · doi:10.1145/3698061.3726934 · Cited by 44
“The field designs, analyzes, and evaluates co-creative AI systems in a wide variety of creative domains [12], such as drawing [11, 16, 19], design [28, 31, 32, 34, 68], dance [36], writing [63], poetry [24, 30, 47], music [2, 26, 27, 37], and video games [22, 40].”
Panchanadikar, R., & Freeman, G. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3677082 · Cited by 43
“Game Development While there has been substantial discourse surrounding the utilization of generative AI in various creative fields [8, 18, 20, 24, 27, 32, 36, 39, 63, 67, 74], research specifically focusing on its application within the game development domain has been notably scarce.”
Shi, J., Jain, R., Chi, S., Doh, H., Chi, H., Quinn, A. J., & Ramani, K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713348 · Cited by 43
Intelligent User Interfaces for Music Discovery
Knees, P., Schedl, M., & Goto, M. (2020) · Transactions of the International Society for Music Information Retrieval · doi:10.5334/tismir.60 · Cited by 43
Effectiveness of AI ‐Driven Vocal Art Tools in Enhancing Student Performance and Creativity
Liu, H., & Guo, W. (2025) · European Journal of Education · doi:10.1111/ejed.70037 · Cited by 43
MusIAC: An Extensible Generative Framework for Music Infilling Applications with Multi-level Control
Guo, R., Simpson, I., Kiefer, C., Magnusson, T., & Herremans, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-03789-4_22 · Cited by 42
Identifying Ethical Issues in AI Partners in Human-AI Co-CreationPreprint
Rezwana, J., & Maher, M. L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.07644 · Cited by 42
“The technological ability of AI alone does not ensure a positive collaborative experience of users with the AI [15].”
BackgroundLoop Copilot: Conducting AI Ensembles for Music Generation and Iterative EditingPreprint
Zhang, Y., Maezawa, A., Xia, G., Yamamoto, K., & Dixon, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.12404 · Cited by 41
“Diving deeper into current popular interactive music interfaces [e.g. 24, 28, 29], while they are powerful and user-friendly, they predominantly focus on a singular type of musical modification: melody inpainting—filling in gaps based on an existing melody.”
BackgroundHuman-Centered AI Communication in Co-Creativity: An Initial Framework and Insights
Rezwana, J., & Ford, C. (2025) · Creativity and Cognition · doi:10.1145/3698061.3726932 · Cited by 40
“AI ability alone does not ensure a positive user experience with the AI [83], especially where interaction is essential between humans and AI [130].”
The Purposeful Presentation of AI Teammates: Impacts on Human Acceptance and Perception
Flathmann, C., Schelble, B. G., McNeese, N. J., Knijnenburg, B. P., Gramopadhye, A. K., & Madathil, K. C. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2254984 · Cited by 39
“…that collaborate with humans (Amershi et al., 2019), and these guidelines have gone on to influence a variety of other human-AI research initiatives, including those that explore trustworthy AI systems (Liao et al., 2020; Shneiderman, 2020a) and even creative AI systems (Louie et al., 2020).”
Shanshan, S., & Sen, G. (2024) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12996 · Cited by 29
Metadata
- Authors
- Louie, R., Coenen, A., Huang, C. Z., Terry, M., & Cai, C. J.
- Affiliations
- Northwestern University (US)
- Google (United States) (US)
- Independent Researcher, Mountain View, CA, USA
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-13
- OpenAlex
- W3031000691
- Semantic Scholar
- 77a5e08f361b6f91cac8a24b380a14c12bb93383
- Cited by
- 271
- References
- 37
- Citations by year
- 2026: 40 · 2025: 75 · 2024: 65 · 2023: 40 · 2022: 34 · 2021: 14 · 2020: 3
- FWCI
- 7.02
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by-sa)
- Topics
- Social Sciences › Arts and Humanities › Music › Music Technology and Sound Studies
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- novice users, generative music, user trust, self-efficacy, human-AI collaboration, music composition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.