Sound Designer-Generative AI Interactions: Towards Designing Creative Support Tools for Professional Sound Designers
Open accessThe DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
The practice of sound design involves creating and manipulating environmental sounds for music, films, or games. Recently, an increasing number of studies have adopted generative AI to assist in sound design co-creation. Most of these studies focus on the needs of novices, and less on the pragmatic needs of sound design practitioners. In this paper, we aim to understand how generative AI models might support sound designers in their practice. We designed two interactive generative AI models as Creative Support Tools (CSTs) and invited nine professional sound design practitioners to apply the CSTs in their practice. We conducted semi-structured interviews and reflected on the challenges and opportunities of using generative AI in mixed-initiative interfaces for sound design. We provide insights into sound designers’ expectations of generative AI and highlight opportunities to situate generative AI-based tools within the design process. Finally, we discuss design considerations for human-AI interaction researchers working with audio.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Tool-mediated passive and semi-proactive agency dominate here.”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [84]
In the citing paper
“Mapping interaction flows reveals prevalent patterns and underexplored areas, such as the sparse use of 'Auditory Interaction' [84, 109, 164], signaling innovation opportunities in domains like assistive technology or music co-creation.”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [84]
In the citing paper
“This evolution may also profoundly alter human cognition and selfperception [55], risking deskilling [202] or over-reliance [84, 104, 107], yet simultaneously offering strong augmentation to human [45].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [84]
Citations
Cites
Reflecting on reflexive thematic analysis
Braun, V., & Clarke, V. (2019) · Qualitative Research in Sport, Exercise and Health · doi:10.1080/2159676x.2019.1628806 · Cited by 19,677
One size fits all? What counts as quality practice in (reflexive) thematic analysis?
Braun, V., & Clarke, V. (2021) · Qualitative Research in Psychology · doi:10.1080/14780887.2020.1769238 · Cited by 9,727
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Braun, V., & Clarke, V. (2021) · Qualitative Research in Sport, Exercise and Health · doi:10.1080/2159676x.2019.1704846 · Cited by 4,776 · cited 2×
“In refexive TA, meaning is not “excavated” [11] from the data, but is subjectively generated through a researcher’s interpretation of the data [9].”
BackgroundMethodGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 1×
“In the last few years, generative models for audio have been applied to creating music [1, 24, 60] and have moved from being exclusively a research endeavor to fnding practical applications [4, 8, 30].”
Background(1994) · Choice Reviews Online · doi:10.5860/choice.32-2038 · Cited by 1,584 · cited 2×
“Typically, sound designers focus on working out the sonic details or timbres (tone or color of the sound) required to enrich or complement the visual information presented in flms and games [19].”
BackgroundAmbiguity as a resource for design
Gaver, W. W., Beaver, J., & Benford, S. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642653 · Cited by 1,474 · cited 2×
“Further, we captured the designer’s interpretations of AI by designing the interactivity with our CSTs using an element of ambiguity [37].”
Shneiderman, B. (2022) · doi:10.1093/oso/9780192845290.001.0001 · Cited by 955
Creativity support tools: accelerating discovery and innovation
Shneiderman, B. (2007) · Communications of the ACM · doi:10.1145/1323688.1323689 · Cited by 765 · cited 2×
“AI algorithms have enabled the building of Creative Support Tools (CSTs) [35, 80] that are either fully autonomous or support co-creation as Mixed-Initiative Creative Interfaces (MICIs) [29, 86].”
Audio Engineering Society Convention
(1985) · The Journal of the Acoustical Society of America · doi:10.1121/1.392015 · Cited by 610
Closed-Form Factorization of Latent Semantics in GANs
Shen, Y., & Zhou, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00158 · Cited by 515 · cited 3×
“For interface-2 , we employed the use of technology-specifc controls [96] based on the SeFa algorithm outlined in [79].”
MethodAudiolm: A language modeling approach to audio generation
Borsos, Z., Marinier, R., Vincent, D., Kharitonov, E., Pietquin, O., Sharifi, M., Roblek, D., Teboul, O., Grangier, D., Tagliasacchi, M., et al. (2023) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2023.3288409 · Cited by 432 · cited 1×
“A similar trend is observed in audio, where a large audio model generates music, environmental sounds, as well as speech [6, 60].”
BackgroundToward a Psychology of Optimal Experience
Csikszentmihalyi, M. (2014) · Flow and the Foundations of Positive Psychology · doi:10.1007/978-94-017-9088-8_14 · Cited by 412 · cited 1×
“…for sound design (section 5.1.4) and fnd that sound designers might overcome this limitation by performing exploration (section 5.2.1) as a separately focused task [26], by employing “reduced listening” (P5) , to “build a library” (P4) of novel sound palettes for use in their projects.”
BackgroundInfluentialMapping the Landscape of Creativity Support Tools in HCI
Frich, J., MacDonald Vermeulen, L., Remy, C., Biskjaer, M. M., & Dalsgaard, P. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300619 · Cited by 410 · cited 2×
“AI algorithms have enabled the building of Creative Support Tools (CSTs) [35, 80] that are either fully autonomous or support co-creation as Mixed-Initiative Creative Interfaces (MICIs) [29, 86].”
Lubart, T. (2005) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2005.04.002 · Cited by 365 · cited 1×
“In [64], Lubart suggests that computers can partner with humans as an enabler, a guide, or a partner or colleague.”
BackgroundOwens, A., Isola, P., McDermott, J., Torralba, A., Adelson, E. H., & Freeman, W. T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.264 · Cited by 346 · cited 1×
“Further, both interfaces provided opportunities to interact with two StyleGANs - (1) one trained on a dataset of ‘Hits & Scratches’ called the Greatest Hits Dataset [73], and (2) another trained on a dataset of ‘Envi-ronmental Sounds’ from the DCASE 2023 Foley Sound Synthesis Challenge [21].”
MethodRezwana, J., & Maher, M. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3519026 · Cited by 342 · cited 1×
“For both interfaces, we adopted an interaction pattern of turn-taking [76], where a user makes modifcations to the control parameters to interact with the underlying AI model and the AI responds based on its trained prior.”
MethodOh, C., Song, J., Choi, J., Kim, S., Lee, S., & Suh, B. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174223 · Cited by 301 · cited 1×
“Interactive CSTs have been studied for their co-creation capabilities in visual arts [17, 27, 43, 53, 71], in writing [14, 23], in fashion [50], in UX design and engineering [38, 54, 63, 67], and in new musical interface design [91].”
BackgroundNovice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models
Louie, R., Coenen, A., Huang, C. Z., Terry, M., & Cai, C. J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376739 · Cited by 271 · cited 4×
“This fnding is in line with some recent research on CSTs in the visual domain, in music composition, and in storytelling where algorithmic tools were used predominantly for idea generation [14, 20, 22, 53, 61, 62].”
InfluentialHertzmann, A. (2018) · Arts · doi:10.3390/arts7020018 · Cited by 265
Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257 · cited 1×
“Interactive CSTs have been studied for their co-creation capabilities in visual arts [17, 27, 43, 53, 71], in writing [14, 23], in fashion [50], in UX design and engineering [38, 54, 63, 67], and in new musical interface design [91].”
BackgroundAtlas of AI: power, politics, and the planetary costs of artificial intelligence.
Can, M. (2022) · International Affairs · doi:10.1093/ia/iiac034 · Cited by 241 · cited 1×
“In light of the recent environmental impact [25] due to the training of large generative AI models, we suggest future CSTs, for all modalities including sound design, could make use of existing pre-trained models by leveraging such post-hoc methods for interactivity.”
MethodDiffsound: Discrete Diffusion Model for Text-to-Sound Generation
Yang, D., Yu, J., Wang, H., Wang, W., Weng, C., Zou, Y., & Yu, D. (2023) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2023.3268730 · Cited by 211 · cited 1×
“Difusion-based models [44, 55, 84, 85, 101] have recently become popular as an alternative to GANs.”
BackgroundLarge-scale Text-to-Image Generation Models for Visual Artists’ Creative Works
Ko, H.-K., Park, G., Jeon, H., Jo, J., Kim, J., & Seo, J. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584078 · Cited by 189 · cited 2×
“This fnding is in line with some recent research on CSTs in the visual domain, in music composition, and in storytelling where algorithmic tools were used predominantly for idea generation [14, 20, 22, 53, 61, 62].”
BackgroundResultUnderstanding the Art of Sound Organization
Landy, L. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7472.001.0001 · Cited by 176
FashionQ: An AI-Driven Creativity Support Tool for Facilitating Ideation in Fashion Design
Jeon, Y., Jin, S., Shih, P. C., & Han, K. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445093 · Cited by 145 · cited 1×
“Interactive CSTs have been studied for their co-creation capabilities in visual arts [17, 27, 43, 53, 71], in writing [14, 23], in fashion [50], in UX design and engineering [38, 54, 63, 67], and in new musical interface design [91].”
BackgroundDavis, N., Hsiao, C.-P., Yashraj Singh, K., Li, L., & Magerko, B. (2016) · Proceedings of the 21st International Conference on Intelligent User Interfaces · doi:10.1145/2856767.2856795 · Cited by 138 · cited 1×
“Interactive CSTs have been studied for their co-creation capabilities in visual arts [17, 27, 43, 53, 71], in writing [14, 23], in fashion [50], in UX design and engineering [38, 54, 63, 67], and in new musical interface design [91].”
AI as Social Glue: Uncovering the Roles of Deep Generative AI during Social Music Composition
Suh, M. (., Youngblom, E., Terry, M., & Cai, C. J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445219 · Cited by 138 · cited 1×
“Further co-creation tools for composing music have been studied for novice co-creation [36, 61, 62, 66, 102] as well as with expert musicians [7, 45, 88].”
Randomness as a resource for design
Leong, T. W., Vetere, F., & Howard, S. (2006) · Proceedings of the 6th conference on Designing Interactive systems · doi:10.1145/1142405.1142428 · Cited by 135 · cited 2×
“In [59], using an example of a music-listening interactive system the authors observe that, at times, unconstrained randomness can contribute to rich user experience (such as serendipity).”
BackgroundGmeiner, F., Yang, H., Yao, L., Holstein, K., & Martelaro, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580999 · Cited by 130 · cited 1×
“Interactive CSTs have been studied for their co-creation capabilities in visual arts [17, 27, 43, 53, 71], in writing [14, 23], in fashion [50], in UX design and engineering [38, 54, 63, 67], and in new musical interface design [91].”
BackgroundMixed-Initiative Creative Interfaces
Deterding, S., Hook, J., Fiebrink, R., Gillies, M., Gow, J., Akten, M., Smith, G., Liapis, A., & Compton, K. (2017) · Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/3027063.3027072 · Cited by 117 · cited 2×
“AI algorithms have enabled the building of Creative Support Tools (CSTs) [35, 80] that are either fully autonomous or support co-creation as Mixed-Initiative Creative Interfaces (MICIs) [29, 86].”
"Explorers of Unknown Planets": Practices and Politics of Artificial Intelligence in Visual Arts
Caramiaux, B., & Fdili Alaoui, S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555578 · Cited by 114 · cited 3×
“Interactive CSTs have been studied for their co-creation capabilities in visual arts [17, 27, 43, 53, 71], in writing [14, 23], in fashion [50], in UX design and engineering [38, 54, 63, 67], and in new musical interface design [91].”
InfluentialDesigning with AI: An Exploration of Co-Ideation with Image Generators
Chiou, L.-Y., Hung, P.-K., Liang, R.-H., & Wang, C.-T. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3596001 · Cited by 113
GenAICHI: Generative AI and HCI
Muller, M., Chilton, L. B., Kantosalo, A., Martin, C. P., & Walsh, G. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503719 · Cited by 101
AI Song Contest: Human-AI Co-Creation in Songwriting
Huang, C.-Z. A., Koops, H. V., Newton-Rex, E., Dinculescu, M., & Cai, C. J. (2020) · arXiv · doi:10.48550/arxiv.2010.05388 · Cited by 90 · cited 1×
“Further co-creation tools for composing music have been studied for novice co-creation [36, 61, 62, 66, 102] as well as with expert musicians [7, 45, 88].”
BackgroundCartwright, M., Seals, A., Salamon, J., Williams, A. C., Mikloska, S., MacConnell, D., Law, E., Bello, J. P., & Nov, O. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134664 · Cited by 83
The Intersection of Users, Roles, Interactions, and Technologies in Creativity Support Tools
Chung, J. J. Y., He, S., & Adar, E. (2021) · Designing Interactive Systems Conference 2021 · doi:10.1145/3461778.3462050 · Cited by 83 · cited 1×
“This fnding is in line with some recent research on CSTs in the visual domain, in music composition, and in storytelling where algorithmic tools were used predominantly for idea generation [14, 20, 22, 53, 61, 62].”
ResultA Noniterative Method for Reconstruction of Phase From STFT Magnitude
Průša, Z., Balázs, P., & Søndergaard, P. L. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2678166 · Cited by 82
When is a Tool a Tool? User Perceptions of System Agency in Human–AI Co-Creative Drawing
Lawton, T., Grace, K., & Ibarrola, F. J. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3595977 · Cited by 72 · cited 1×
“Therefore, although interface-2 opened up more personalized avenues for the designers to interact with the AI, the ambiguity in the dimensions got in the way of its agentive fow [57], a highly engaging state of interacting with an AI-based CST.”
BackgroundThe Machine Learning Algorithm as Creative Musical Tool
Fiebrink, R., & Caramiaux, B. (2016) · arXiv · doi:10.48550/arxiv.1611.00379 · Cited by 69 · cited 1×
“In the feld of audio, machine learning models have long been used for creating music [32], from established tools such as Wekinator [33] to the more recent GAN-based music performance art [90].”
MethodSemanticCollage: Enriching Digital Mood Board Design with Semantic Labels
Koch, J., Taffin, N., Lucero, A., & Mackay, W. E. (2020) · Proceedings of the 2020 ACM Designing Interactive Systems Conference · doi:10.1145/3357236.3395494 · Cited by 63 · cited 1×
“Interactive CSTs have been studied for their co-creation capabilities in visual arts [17, 27, 43, 53, 71], in writing [14, 23], in fashion [50], in UX design and engineering [38, 54, 63, 67], and in new musical interface design [91].”
BackgroundBridging the Gap Between UX Practitioners’ Work Practices and AI-Enabled Design Support Tools
Lu, Y., Zhang, C., Zhang, I., & Li, T. J.-J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519809 · Cited by 63 · cited 1×
“Interactive CSTs have been studied for their co-creation capabilities in visual arts [17, 27, 43, 53, 71], in writing [14, 23], in fashion [50], in UX design and engineering [38, 54, 63, 67], and in new musical interface design [91].”
BackgroundFrid, E., Gomes, C., & Jin, Z. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376514 · Cited by 58 · cited 2×
“Most human-AI interaction studies for audio focus on the applicability of such steerable interfaces to empower novice users in their creative goals [7, 36, 61, 62, 102].”
BackgroundA Generative Model for Raw Audio Using Transformer Architectures
Verma, P., & Chafe, C. (2021) · International Conference on Digital Audio Effects (DAFx) · doi:10.23919/dafx51585.2021.9768298 · Cited by 58
Toward General Design Principles for Generative AI Applications
Weisz, J. D., Muller, M., He, J., & Houde, S. (2023) · arXiv · doi:10.48550/arxiv.2301.05578 · Cited by 54
Sound design: an applied, experimental framework to study the perception of everyday sounds
Susini, P., Houix, O., & Misdariis, N. (2014) · The New Soundtrack · doi:10.3366/sound.2014.0057 · Cited by 43 · cited 8×
“They also focus on communicating additional non-verbal information through interactions in games or product design [89].”
InfluentialExplainable AI for the Arts: XAIxArts
Bryan–Kinns, N., Ford, C., Chamberlain, A., Benford, S., Kennedy, H., Li, Z., Wu, Q., Xia, G., & Rezwana, J. (2023) · Creativity and Cognition · doi:10.1145/3591196.3593517 · Cited by 43 · cited 2×
“…the individual events” (P2) , “fade-in is quite long” (P4) , or “removing the initial transient and softening it to leave the body and tail” (P4) , etc. Previously, researchers in the explainable AI (XAI) for arts [12, 13] used latent space visualizations to explain or debug their creative goals.”
Practice-led strategies for interactive art research
Candy, L., Amitani, S., & Bilda, Z. (2006) · CoDesign · doi:10.1080/15710880601007994 · Cited by 40 · cited 1×
“This approach was adapted from the video-cue recall method [15, 16] from the interactive arts literature for our purpose.”
Designing Deep Reinforcement Learning for Human Parameter Exploration
Scurto, H., Kerrebroeck, B. V., Caramiaux, B., & Bevilacqua, F. P. (2021) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3414472 · Cited by 40 · cited 1×
“In the sound design space, Scurto et al. [77] developed tools based on reinforcement learning algorithms and studied user exploration behaviors for the generated high-dimensional parameter spaces.”
GenAICHI 2023: Generative AI and HCI at CHI 2023
Muller, M. J., Chilton, L. B., Kantosalo, A., Liao, Q. V., Maher, M. L., Martin, C. P., & Walsh, G. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3573794 · Cited by 34 · cited 1×
“Recently, human-AI interaction researchers have been increasingly interested in understanding how mixed-initiative creative interfaces (MICIs) [29] can be applied in a work setting in diferent domains of creative work [68, 69, 96].”
BackgroundMozaffari, M. A., Zhang, X., Cheng, J., & Guo, J. L. C. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517511 · Cited by 33
pyloudnorm: A simple yet flexible loudness meter in Python
Steinmetz, C. J., & Reiss, J. D. (2021) · Queen Mary Research Online (Queen Mary University of London) · Cited by 31
Louie, R., Engel, J., & Huang, C.-Z. A. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511159 · Cited by 30 · cited 4×
“This fnding is in line with some recent research on CSTs in the visual domain, in music composition, and in storytelling where algorithmic tools were used predominantly for idea generation [14, 20, 22, 53, 61, 62].”
BackgroundResultInfluentialInteractive Exploration-Exploitation Balancing for Generative Melody Composition
Zhou, Y., Koyama, Y., Goto, M., & Igarashi, T. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450663 · Cited by 24 · cited 2×
“Most human-AI interaction studies for audio focus on the applicability of such steerable interfaces to empower novice users in their creative goals [7, 36, 61, 62, 102].”
BackgroundCollaborating with an Autonomous Agent to Generate Affective Music
Morreale, F., & Angeli, A. D. (2016) · Computers in entertainment · doi:10.1145/2967508 · Cited by 23 · cited 1×
“Further co-creation tools for composing music have been studied for novice co-creation [36, 61, 62, 66, 102] as well as with expert musicians [7, 45, 88].”
BackgroundSelf-citationTowards a conceptual framework to integrate designerly and scientific sound design methods
Hug, D., & Misdariis, N. (2011) · Audio Mostly Conference (AM) · doi:10.1145/2095667.2095671 · Cited by 21 · cited 2×
“These modes of working help them develop ’ Sonic Vocabularies ’ [47] and ’ Sound Palettes ’ for various current and future projects.”
BackgroundMethodDeep Generative Models for Musical Audio Synthesis
Huzaifah, M., & Wyse, L. L. (2021) · Springer eBooks · doi:10.1007/978-3-030-72116-9_22 · Cited by 21
Calliope: A Co-creative Interface for Multi-Track Music Generation
Tchemeube, R. B., Ens, J., & Pasquier, P. (2022) · Creativity and Cognition · doi:10.1145/3527927.3535200 · Cited by 19 · cited 2×
“Most human-AI interaction studies for audio focus on the applicability of such steerable interfaces to empower novice users in their creative goals [7, 36, 61, 62, 102].”
BackgroundSound Design Theory and Practice
Murray, L. (2019) · Routledge eBooks · doi:10.4324/9781315647517 · Cited by 19
Digital Musical Instruments as Research Products
Jack, R., Harrison, J., & McPherson, A. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.4813464 · Cited by 18
Man, B. D., Stables, R., & Reiss, J. D. (2019) · Focal Press eBooks · doi:10.4324/9781315166100 · Cited by 17
Tahıroğlu, K., Kastemaa, M., & Koli, O. (2020) · Aaltodoc (Aalto University) · doi:10.5281/zenodo.4813401 · Cited by 15
Learning from the Veg Box: Designing Unpredictability in Agency Delegation
Verame, J. K. M., Costanza, E., Fischer, J., Crabtree, A., Ramchurn, S. D., Rodden, T., & Jennings, N. R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174021 · Cited by 13 · cited 1×
“Previously, researchers have investigated the role of constrained and unconstrained randomness in interactive systems on user experience [59, 93].”
BackgroundTrends in Audio Texture Analysis, Synthesis, and Applications
Sharma, G., & Umapathy, K. (2022) · Journal of the Audio Engineering Society · doi:10.17743/jaes.2021.0060 · Cited by 10
Towards Controllable Audio Texture Morphing
Gupta, C., Kamath, P., Wei, Y., Li, Z., Nanayakkara, S. C., & Wyse, L. L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096328 · Cited by 9 · cited 1×
“Such models are expressive in their ability to allow the generation of novel sounds or morphs [41] as compared to autoregressive architectures [100].”
BackgroundSelf-citationSound Design Theory and Practice: Working with Sound
Murray, L. (2019) · Murdoch University Research Portal · Cited by 6
Proceedings of the 18th Sound and Music Computing Conference
Mauro, D. A., Spagnol, S., & Valle, A. (2021) · Institutional Research Information System University of Turin (University of Turin) · doi:10.5281/zenodo.5113511 · Cited by 4
From Mondrian to Modular Synth: Rendering NIME using Generative Adversarial Networks
van Troyer, A., & Kleinberger, R. (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3672955 · Cited by 2
Research works printed but not linked: 31
- Andrea Agostinelli, Timo I Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, 2023. Musiclm: Generating music from text. arXiv preprint arXiv:2301.11325 (2023). · arXiv
- Open AI. 2023. Dall.E 2. https://openai.com/dall-e-2 [Accessed: 29 August 2023].
- Aimi.Fm. 2023. Aimi.Fm. https://www.aimi.fm/ [Accessed: 15 November 2023].
- brain.fm. 2023. brain.fm. https://www.brain.fm/ [Accessed: 15 November 2023].
- Nick Bryan-Kinns, Berker Banar, Corey Ford, Courtney N. Reed, Yixiao Zhang, Simon Colton, and Jack Armitage. 2021. Exploring XAI for the Arts: Explaining Latent Space in Generative Music, In eXplainable AI approaches for debugging and diagnosis.XAI 4 Debugging Workshop at NEURIPS 2021. https://openreview.net/forum?id=GLhY_0xMLZr
- Linda Candy. 2006. Practice based research: A guide. CCS report 1, 2 (2006), 1–19.
- Keunwoo Choi, Jaekwon Im, Laurie Heller, Brian McFee, Keisuke Imoto, Yuki Okamoto, Mathieu Lagrange, and Shinosuke Takamichi. 2023. Foley sound synthesis at the dcase 2023 challenge. arXiv preprint arXiv:2304.12521 (2023). · arXiv
- Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Défossez. 2023. Simple and Controllable Music Generation. In Thirty-seventh Conference on Neural Information Processing Systems. https://openreview.net/forum?id=jtiQ26sCJi
- Brecht De Man, Ryan Stables, and Joshua D Reiss. 2019. Intelligent Music Production. Routledge.
- Endel. 2023. Endel. https://endel.io/ [Accessed: 15 November 2023].
- Rebecca Fiebrink and Perry R Cook. 2010. The Wekinator: a system for real-time, interactive machine learning in music. In Proceedings of The Eleventh International Society for Music Information Retrieval Conference (ISMIR 2010)(Utrecht), Vol. 3. Citeseer, 2–1.
- Corey Ford and Nick Bryan-Kinns. 2022. Speculating on Reflection and People’s Music Co-Creation with AI. Workshop on Generative AI and HCI at the CHI Conference on Human Factors in Computing Systems 2022 (2022).
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems 27 (2014).
- Ian J. Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press, Cambridge, MA, USA. http://www.deeplearningbook.org.
- Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., 6840–6851. https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf
- Pytorch Hub. 2023. Pytorch Hub. https://pytorch.org/hub/ [Accessed: 15 November 2023].
- Purnima Kamath Chitralekha Gupta Lonce Wyse and Suranga Nanayakkara. 2023. Example-Based Framework for Perceptually Guided Audio Texture Generation. arxiv:2308.11859 [eess.AS] · arXiv
- Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. 2021. DiffWave: A Versatile Diffusion Model for Audio Synthesis. In International Conference on Learning Representations. https://openreview.net/forum?id=a-xFK8Ymz5J
- Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. 2023. AudioLDM: Text-to-Audio Generation with Latent Diffusion Models. In Proceedings of the 40th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 202), Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.). PMLR, 21450–21474. https://proceedings.mlr.press/v202/liu23f.html
- Andrés Marafioti, Nathanaël Perraudin, Nicki Holighaus, and Piotr Majdak. 2019. Adversarial Generation of Time-Frequency Features with application in audio synthesis. In Proceedings of the 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 4352–4362. https://proceedings.mlr.press/v97/marafioti19a.html
- Sangshin Oh Minsung Kang Hyeongi Moon Keunwoo Choi and Ben Sangbae Chon. 2023. A Demand-Driven Perspective on Generative Audio AI. arxiv:2307.04292 [eess.AS] · arXiv
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.). Vol. 32. Curran Associates, Inc.https://proceedings.neurips.cc/paper_files/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf
- Denis Smalley. 1997. Spectromorphology: explaining sound-shapes. Organised sound 2, 2 (1997), 107–126.
- 10.5555/3294996.3295163 · doi:10.5555/3294996.3295163
- Jiaming Song, Chenlin Meng, and Stefano Ermon. 2021. Denoising Diffusion Implicit Models. In International Conference on Learning Representations. https://openreview.net/forum?id=St1giarCHLP
- Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. 2021. Maximum Likelihood Training of Score-Based Diffusion Models. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.). Vol. 34. Curran Associates, Inc., 1415–1428. https://proceedings.neurips.cc/paper_files/paper/2021/file/0a9fdbb17feb6ccb7ec405cfb85222c4-Paper.pdf
- Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu. 2016. WaveNet: A Generative Model for Raw Audio. CoRR abs/1609.03499 (2016). arXiv:1609.03499http://arxiv.org/abs/1609.03499 · arXiv
- Graham Wallas. 1926. The art of thought. Vol. 10. Harcourt Brace.
- Lonce Wyse. 2018. Real-valued parametric conditioning of an RNN for interactive sound synthesis. arXiv preprint arXiv:1805.10808 (2018). · arXiv
- Lonce Wyse, Purnima Kamath, and Chitralekha Gupta. 2022. Sound Model Factory: An Integrated System Architecture for Generative Audio Modelling. In Artificial Intelligence in Music, Sound, Art and Design, Tiago Martins, Nereida Rodríguez-Fernández, and Sérgio M. Rebelo (Eds.). Springer International Publishing, Cham, 308–322.
- Model Zoo. 2023. Model Zoo. https://modelzoo.co/ [Accessed: 15 November 2023].
Cited by
Zhang, S., Wang, H., & Yi, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757594 · Cited by 36
“…or next-paragraph suggestions to steer the writing process [45], generative AI tools initiating the creation of novel sound options for designers [84], and interactive tools proactively highlighting relevant news or generating specific use cases to make users consider potential AI harms early on…”
Influential(19448503), J. H., Hitchen, G., & Dogan, S. (2025) · Convergence The International Journal of Research into New Media Technologies · doi:10.1177/13548565251383409 · Cited by 32
“Prototypes in fi lm explore generative AI ’ s capabilities in sound design (Kamath et al., 2024), face swapping (Jia and Yang, 2022), and storytelling (Briggs and Pledger, 2024), showcasing its contributions to different production stages.”
Amuse: Human-AI Collaborative Songwriting with Multimodal Inspirations
Kim, Y., Lee, S.-J., & Donahue, C. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713818 · Cited by 23
“We hypothesize that the Chord Generator, by accepting non-musical inputs such as images and texts, offered the creative freedom necessary to compose songs based on songwriting prompts (e.g., “Write an 8-bar chorus about your favorite summer holiday memory.”)”
GenAICHI 2025: Generative AI and HCI at CHI 2025
Muller, M. J., Chilton, L. B., Maher, M. L., Martin, C. P., Choi, M., Walsh, G., & Kantosalo, A. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3707213 · Cited by 23
Exploring the Potential of Music Generative AI for Music-Making by Deaf and Hard of Hearing People
Choi, Y., Moon, J., Yoo, J., & Hong, J.-H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714298 · Cited by 19
“Previous studies have demonstrated the potential of music GenAI for helping professional composers explore their musical ideas, as well as supporting the composition efforts of novices with relatively little experience [4, 7, 29, 38, 89].”
Metadata
- Authors
- Kamath, P., Morreale, F., Bagaskara, P. L., Wei, Y., & Nanayakkara, S.
- Affiliations
- National University of Singapore (SG)
- University of Auckland (NZ)
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-17
- OpenAlex
- W4396833556
- Semantic Scholar
- d35a72f791f8cfe11b17961a33b3bd3bbcddb6b4
- Cited by
- 37
- References
- 64
- Citations by year
- 2026: 14 · 2025: 19 · 2024: 4
- FWCI
- 7.64
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by-sa)
- Topics
- Social Sciences › Arts and Humanities › Music › Music Technology and Sound Studies
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- sound design, generative AI, human-AI interaction, audio, design process
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.