CoChat
Sign Up
PreprintCited by 69Music Technology and Sound StudiesArt, Technology, and CultureMusicology and Musical Analysis

The Machine Learning Algorithm as Creative Musical Tool

Open access
Fiebrink, Rebecca, Caramiaux, Baptiste·2016·arXiv
DOI 10.48550/arxiv.1611.00379arXiv 1611.00379
Open this paper in CoChat — free accountOpen PDF

Abstract

Machine learning is the capacity of a computational system to learn structures from datasets in order to make predictions on newly seen data. Such an approach offers a significant advantage in music scenarios in which musicians can teach the system to learn an idiosyncratic style, or can break the rules to explore the system's capacity in unexpected ways. In this chapter we draw on music, machine learning, and human-computer interaction to elucidate an understanding of machine learning algorithms as creative tools for music and the sonic arts. We motivate a new understanding of learning algorithms as human-computer interfaces. We show that, like other interfaces, learning algorithms can be characterised by the ways their affordances intersect with goals of human users. We also argue that the nature of interaction between users and algorithms impacts the usability and usefulness of those algorithms in profound ways. This human-centred view of machine learning motivates our concluding discussion of what it means to employ machine learning as a creative tool.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1611.00379

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 55

  1. Akrour, R., Schoenauer, M., Sebag, M., and Souplet, J.-C. (2014). Programming by feedback. In International Conference on Machine Learning , number 32, pages 15031511. JMLR. org.
  2. Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., and Suh, J. (2015). Modeltracker: Redesigning performance analysis tools for machine learning. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems , pages 337-346. ACM.
  3. Amershi, S., Fogarty, J., and Weld, D. (2012). Regroup: Interactive machine learning for on-demand group creation in social networks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , pages 21-30. ACM.
  4. Ames, C. (1989). The markov process as a compositional model: A survey and tutorial. Leonardo , pages 175-187.
  5. Assayag, G., Bloch, G., Chemillier, M., Cont, A., and Dubnov, S. (2006). Omax brothers: a dynamic yopology of agents for improvization learning. In Proceedings of the 1st ACM workshop on Audio and music computing multimedia , pages 125-132. ACM.
  6. Assayag, G. and Dubnov, S. (2004). Using factor oracles for machine improvisation. Soft Computing , 8(9):604-610.
  7. Bevilacqua, F., Schnell, N., Rasamimanana, N., Zamborlin, B., and Gu´ edy, F. (2011). Online gesture analysis and control of audio processing. In Musical Robots and Interactive Multimodal Systems , pages 127-142. Springer.
  8. Bevilacqua, F., Zamborlin, B., Sypniewski, A., Schnell, N., Gu´ edy, F., and Rasamimanana, N. (2010). Continuous realtime gesture following and recognition. In Communication and Human-Computer Interaction, Volume 5934 of Lecture Notes in Computer Science , pages 73-84. Springer.
  9. Bullock, J. (2017). Designing interfaces for musical algorithms. In Oxford Handbook of Algorithmic Music . Oxford University Press Oxford, UK.
  10. Buxton, B. (2010). Sketching user experiences: getting the design right and the right design . Morgan Kaufmann.
  11. Caramiaux, B., Altavilla, A., Pobiner, S. G., and Tanaka, A. (2015). Form follows sound: Designing interactions from sonic memories. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems , pages 3943-3952. ACM.
  12. Caramiaux, B., Fran¸ coise, J., Schnell, N., and Bevilacqua, F. (2014a). Mapping through listening. Computer Music Journal , 38(3):34-48.
  13. Caramiaux, B., Montecchio, N., Tanaka, A., and Bevilacqua, F. (2014b). Adaptive gesture recognition with variation estimation for interactive systems. ACM Transactions on Interactive Intelligent Systems (TiiS) , 4(4):18.
  14. Caramiaux, B., Wanderley, M. M., and Bevilacqua, F. (2012). Segmenting and parsing instrumentalists' gestures. Journal of New Music Research , 41(1):13-29.
  15. Conklin, D. (2003). Music generation from statistical models. In Proceedings of the AISB 2003 Symposium on Artificial Intelligence and Creativity in the Arts and Sciences , pages 30-35. Citeseer.
  16. Cont, A. (2010). A coupled duration-focused architecture for real-time music-to-score alignment. IEEE transactions on pattern analysis and machine intelligence , 32(6):974987.
  17. Csikszentmihalyi, M. and Csikszentmihalyi, I. S. (1992). Optimal experience: Psychological studies of flow in consciousness . Cambridge university press.
  18. Drummond, J. (2009). Understanding interactive systems. Organised Sound , 14(02):124133.
  19. Fails, J. A. and Olsen Jr, D. R. (2003). Interactive machine learning. In Proceedings of the 8th international conference on Intelligent user interfaces , pages 39-45. ACM.
  20. Fasciani, S. and Wyse, L. (2013). A self-organizing gesture map for a voice-controlled instrument interface. In NIME , pages 507-512.
  21. Fiebrink, R., Cook, P. R., and Trueman, D. (2011). Human model evaluation in interactive supervised learning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , pages 147-156. ACM.
  22. Fiebrink, R., Trueman, D., and Cook, P. R. (2009b). A meta-instrument for interactive, on-the-fly machine learning. In NIME , pages 280-285.
  23. Fran¸ coise, J. (2015). Motion-Sound Mapping by Demonstration . PhD thesis, UPMC.
  24. Fran¸ coise, J., Caramiaux, B., and Bevilacqua, F. (2012). A hierarchical approach for the design of gesture-to-sound mappings. In 9th Sound and Music Computing Conference , pages 233-240.
  25. Fran¸ coise, J., Schnell, N., and Bevilacqua, F. (2013). A multimodal probabilistic model for gesture-based control of sound synthesis. In Proceedings of the 21st ACM international conference on Multimedia , pages 705-708. ACM.
  26. Fried, O. and Fiebrink, R. (2013). Cross-modal sound mapping using deep learning. In NIME , pages 531-534.
  27. Fried, O., Jin, Z., Oda, R., and Finkelstein, A. (2014). Audioquilt: 2d arrangements of audio samples using metric learning and kernelized sorting. In NIME , pages 281-286.
  28. Gaver, W. W. (1991). Technology affordances. In Proceedings of the SIGCHI conference on Human factors in computing systems , pages 79-84. ACM.
  29. Groce, A., Kulesza, T., Zhang, C., Shamasunder, S., Burnett, M., Wong, W.-K., Stumpf, S., Das, S., Shinsel, A., Bice, F., et al. (2014). You are the only possible oracle: Effective test selection for end users of interactive machine learning systems. IEEE Transactions on Software Engineering , 40(3):307-323.
  30. Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A.-r., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., et al. (2012). Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine , 29(6):82-97.
  31. Hiraga, R., Bresin, R., Hirata, K., and Katayose, H. (2004). Rencon 2004: Turing test for musical expression. In Proceedings of the 2004 conference on New interfaces for musical expression , pages 120-123. National University of Singapore.
  32. Humphrey, E. J., Bello, J. P., and LeCun, Y. (2012). Moving beyond feature design: Deep architectures and automatic feature learning in music informatics. In ISMIR , pages 403-408. Citeseer.
  33. Jordan, M. (2011). Message from the president: The era of big data. ISBA Bulletin , 18(2):1-3.
  34. Khan, F., Mutlu, B., and Zhu, X. (2011). How do humans teach: On curriculum learning and teaching dimension. In Advances in Neural Information Processing Systems , pages 1449-1457.
  35. Kulesza, T., Stumpf, S., Wong, W.-K., Burnett, M. M., Perona, S., Ko, A., and Oberst, I. (2011). Why-oriented end-user debugging of naive bayes text classification. ACM Transactions on Interactive Intelligent Systems (TiiS) , 1(1):2.
  36. Le, Q. V. (2013). Building high-level features using large scale unsupervised learning. In 2013 IEEE international conference on acoustics, speech and signal processing , pages 8595-8598. IEEE.
  37. Lee, M., Freed, A., and Wessel, D. (1991). Real-time neural network processing of gestural and acoustic signals. In Proceedings of the International Computer Music Conference , pages 277-277. INTERNATIONAL COMPUTER MUSIC ACCOCIATION.
  38. Leman, M. (2008). Embodied music cognition and mediation technology . Mit Press.
  39. Lippe, C. (2002). Real-time interaction among composers, performers, and computer systems. Information Processing Society of Japan SIG Notes , 2002(123):1-6.
  40. McGrenere, J. and Ho, W. (2000). Affordances: Clarifying and evolving a concept. In Graphics interface , volume 2000, pages 179-186.
  41. Mobahi, H., Collobert, R., and Weston, J. (2009). Deep learning from temporal coherence in video. In Proceedings of the 26th Annual International Conference on Machine Learning , pages 737-744. ACM.
  42. Morris, D., Simon, I., and Basu, S. (2008). Exposing parameters of a trained dynamic model for interactive music creation. In AAAI , pages 784-791.
  43. Murphy, K. P. (2012). Machine learning: a probabilistic perspective . MIT press.
  44. Pachet, F. (2003). The continuator: Musical interaction with style. Journal of New Music Research , 32(3):333-341.
  45. Pachet, F. (2008). The future of content is in ourselves. Computers in Entertainment (CIE) , 6(3):31.
  46. Pardo, B., Little, D., and Gergle, D. (2012). Building a personalized audio equalizer interface with transfer learning and active learning. In Proceedings of the second international ACM workshop on Music information retrieval with user-centered and multimodal strategies , pages 13-18. ACM.
  47. Rabiner, L. R. (1989). A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE , 77(2):257-286.
  48. Raphael, C. (2001). Synthesizing musical accompaniments with bayesian belief networks. Journal of New Music Research , 30(1):59-67.
  49. Rowe, R. (1992). Interactive music systems: machine listening and composing . MIT press.
  50. Sarkar, M. and Vercoe, B. (2007). Recognition and prediction in a network music performance system for indian percussion. In Proceedings of the 7th international conference on new interfaces for musical expression , pages 317-320. ACM.
  51. Stowell, D. (2010). Making music through real-time voice timbre analysis: machine learning and timbral control . PhD thesis.
  52. Wanderley, M. M. and Depalle, P. (2004). Gestural control of sound synthesis. Proceedings of the IEEE , 92(4):632-644.
  53. Wilson, A. D. and Bobick, A. F. (2000). Realtime online adaptive gesture recognition. In Pattern Recognition, 2000. Proceedings. 15th International Conference on , volume 1, pages 270-275. IEEE.
  54. Witten, I. H. and Frank, E. (2005). Data Mining: Practical machine learning tools and techniques . Morgan Kaufmann.
  55. Zamborlin, B., Bevilacqua, F., Gillies, M., and D'inverno, M. (2014). Fluid gesture interaction design: Applications of continuous recognition for the design of modern gestural interfaces. ACM Transactions on Interactive Intelligent Systems (TiiS) , 3(4):22.

Cited by

Sorted by citations

Authors
Fiebrink, R., & Caramiaux, B.
Affiliations
University of London (GB)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2016
DOI
10.48550/arxiv.1611.00379
arXiv
1611.00379
OpenAlex
W2546879872
Cited by
69
References
0
Citations by year
2026: 6 · 2025: 8 · 2024: 4 · 2023: 15 · 2022: 8 · 2021: 12 · 2020: 4 · 2019: 3 · 2018: 5 · 2017: 2 · 2016: 2
FWCI
4.50
Type
book
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • Oxford University Press eBooks (publishedVersion)
  • Goldsmiths (University of London) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Music Technology and Sound Studies
  • Social Sciences › Arts and Humanities › Visual Arts and Performing Arts › Art, Technology, and Culture
  • Social Sciences › Arts and Humanities › Music › Musicology and Musical Analysis
Keywords
machine learning algorithms, compositional process, human-computer interaction, affordances, human-centered design, usability, sonic art, algorithmic creativity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review