Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.

Sound Designer-Generative AI Interactions: Towards Designing Creative Support Tools for Professional Sound Designers

Open access
Kamath, Purnima, Morreale, Fabio, Bagaskara, Priambudi Lintang, Wei, Yize, Nanayakkara, Suranga · 2024 · Proceedings of the CHI Conference on Human Factors in Computing Systems

Abstract

The practice of sound design involves creating and manipulating environmental sounds for music, films, or games. Recently, an increasing number of studies have adopted generative AI to assist in sound design co-creation. Most of these studies focus on the needs of novices, and less on the pragmatic needs of sound design practitioners. In this paper, we aim to understand how generative AI models might support sound designers in their practice. We designed two interactive generative AI models as Creative Support Tools (CSTs) and invited nine professional sound design practitioners to apply the CSTs in their practice. We conducted semi-structured interviews and reflected on the challenges and opportunities of using generative AI in mixed-initiative interfaces for sound design. We provide insights into sound designers’ expectations of generative AI and highlight opportunities to situate generative AI-based tools within the design process. Finally, we discuss design considerations for human-AI interaction researchers working with audio.

Overview

Integrity signals

Registry

Crossref · 10.1145/3613904.3642040

Cites

67 of 67 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 64 references

Research works printed but not linked: 32

  1. Andrea Agostinelli, Timo I Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, 2023. Musiclm: Generating music from text. arXiv preprint arXiv:2301.11325 (2023). · arXiv
  2. Open AI. 2023. Dall.E 2. https://openai.com/dall-e-2 [Accessed: 29 August 2023].
  3. Aimi.Fm. 2023. Aimi.Fm. https://www.aimi.fm/ [Accessed: 15 November 2023].
  4. brain.fm. 2023. brain.fm. https://www.brain.fm/ [Accessed: 15 November 2023].
  5. Nick Bryan-Kinns, Berker Banar, Corey Ford, Courtney N. Reed, Yixiao Zhang, Simon Colton, and Jack Armitage. 2021. Exploring XAI for the Arts: Explaining Latent Space in Generative Music, In eXplainable AI approaches for debugging and diagnosis.XAI 4 Debugging Workshop at NEURIPS 2021. https://openreview.net/forum?id=GLhY_0xMLZr
  6. Linda Candy. 2006. Practice based research: A guide. CCS report 1, 2 (2006), 1–19.
  7. Keunwoo Choi, Jaekwon Im, Laurie Heller, Brian McFee, Keisuke Imoto, Yuki Okamoto, Mathieu Lagrange, and Shinosuke Takamichi. 2023. Foley sound synthesis at the dcase 2023 challenge. arXiv preprint arXiv:2304.12521 (2023). · arXiv
  8. Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Défossez. 2023. Simple and Controllable Music Generation. In Thirty-seventh Conference on Neural Information Processing Systems. https://openreview.net/forum?id=jtiQ26sCJi
  9. Brecht De Man, Ryan Stables, and Joshua D Reiss. 2019. Intelligent Music Production. Routledge.
  10. Endel. 2023. Endel. https://endel.io/ [Accessed: 15 November 2023].
  11. Rebecca Fiebrink and Perry R Cook. 2010. The Wekinator: a system for real-time, interactive machine learning in music. In Proceedings of The Eleventh International Society for Music Information Retrieval Conference (ISMIR 2010)(Utrecht), Vol. 3. Citeseer, 2–1.
  12. Corey Ford and Nick Bryan-Kinns. 2022. Speculating on Reflection and People’s Music Co-Creation with AI. Workshop on Generative AI and HCI at the CHI Conference on Human Factors in Computing Systems 2022 (2022).
  13. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems 27 (2014).
  14. Ian J. Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press, Cambridge, MA, USA. http://www.deeplearningbook.org.
  15. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., 6840–6851. https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf
  16. Pytorch Hub. 2023. Pytorch Hub. https://pytorch.org/hub/ [Accessed: 15 November 2023].
  17. Purnima Kamath Chitralekha Gupta Lonce Wyse and Suranga Nanayakkara. 2023. Example-Based Framework for Perceptually Guided Audio Texture Generation. arxiv:2308.11859 [eess.AS] · arXiv
  18. Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. 2021. DiffWave: A Versatile Diffusion Model for Audio Synthesis. In International Conference on Learning Representations. https://openreview.net/forum?id=a-xFK8Ymz5J
  19. Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. 2023. AudioLDM: Text-to-Audio Generation with Latent Diffusion Models. In Proceedings of the 40th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 202), Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.). PMLR, 21450–21474. https://proceedings.mlr.press/v202/liu23f.html
  20. Andrés Marafioti, Nathanaël Perraudin, Nicki Holighaus, and Piotr Majdak. 2019. Adversarial Generation of Time-Frequency Features with application in audio synthesis. In Proceedings of the 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 4352–4362. https://proceedings.mlr.press/v97/marafioti19a.html
  21. Sangshin Oh Minsung Kang Hyeongi Moon Keunwoo Choi and Ben Sangbae Chon. 2023. A Demand-Driven Perspective on Generative Audio AI. arxiv:2307.04292 [eess.AS] · arXiv
  22. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.). Vol. 32. Curran Associates, Inc.https://proceedings.neurips.cc/paper_files/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf
  23. Denis Smalley. 1997. Spectromorphology: explaining sound-shapes. Organised sound 2, 2 (1997), 107–126.
  24. 10.5555/3294996.3295163 · doi:10.5555/3294996.3295163
  25. Jiaming Song, Chenlin Meng, and Stefano Ermon. 2021. Denoising Diffusion Implicit Models. In International Conference on Learning Representations. https://openreview.net/forum?id=St1giarCHLP
  26. Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. 2021. Maximum Likelihood Training of Score-Based Diffusion Models. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.). Vol. 34. Curran Associates, Inc., 1415–1428. https://proceedings.neurips.cc/paper_files/paper/2021/file/0a9fdbb17feb6ccb7ec405cfb85222c4-Paper.pdf
  27. Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu. 2016. WaveNet: A Generative Model for Raw Audio. CoRR abs/1609.03499 (2016). arXiv:1609.03499http://arxiv.org/abs/1609.03499 · arXiv
  28. Graham Wallas. 1926. The art of thought. Vol. 10. Harcourt Brace.
  29. Justin D. Weisz, Michael J. Muller, Jessica He, and Stephanie Houde. 2023. Toward General Design Principles for Generative AI Applications 130-144. In IUI Workshops. https://api.semanticscholar.org/CorpusID:255825625
  30. Lonce Wyse. 2018. Real-valued parametric conditioning of an RNN for interactive sound synthesis. arXiv preprint arXiv:1805.10808 (2018). · arXiv
  31. Lonce Wyse, Purnima Kamath, and Chitralekha Gupta. 2022. Sound Model Factory: An Integrated System Architecture for Generative Audio Modelling. In Artificial Intelligence in Music, Sound, Art and Design, Tiago Martins, Nereida Rodríguez-Fernández, and Sérgio M. Rebelo (Eds.). Springer International Publishing, Cham, 308–322.
  32. Model Zoo. 2023. Model Zoo. https://modelzoo.co/ [Accessed: 15 November 2023].

Cited by

See all 47

Metadata

Authors
Kamath, P., Morreale, F., Bagaskara, P. L., Wei, Y., & Nanayakkara, S.
Affiliations
  • National University of Singapore (SG)
  • University of Auckland (NZ)
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-17
OpenAlex
W4396833556
Cited by
37
References
64
Citations by year
2026: 14 · 2025: 19 · 2024: 4
FWCI
7.64
Type
conference paper
Language
en
Open access
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by-sa)
Topics
  • Social Sciences › Arts and Humanities › Music › Music Technology and Sound Studies
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords
sound design, generative AI, human-AI interaction, audio, design process

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Sound Designer-Generative AI Interactions: Towards Designing Creative Support Tools for Professional Sound Designers. https://app.cochat.ai/papers/10.1145/3613904.3642040
The paper
Kamath, P., Morreale, F., Bagaskara, P. L., Wei, Y., & Nanayakkara, S. (2024). Sound Designer-Generative AI Interactions: Towards Designing Creative Support Tools for Professional Sound Designers. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642040

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.