Abstract
This paper presents a multidisciplinary case study of practice with machine learning for computer music. It builds on the scientific study of two machine learning models respectively developed for data-driven sound synthesis and interactive exploration. It details how the learning capabilities of the two models were leveraged to design and implement a musical interface focused on embodied musical interaction. It then describes how this interface was employed and applied to the composition and performance of ægo, an improvisational piece with interactive sound and image for one performer. We discuss the outputs of our research and creation process, and expose our personal reflections and insights on transdisciplinary research opportunities framed by machine learning for computer music.
H. Scurto and A. Chemla-Romeu-Santos—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
See these video excerpts from early rehearsals: https://vimeo.com/418787133.
References
Akten, M., Fiebrink, R., Grierson, M.: Deep meditations: controlled navigation of latent space. Goldsmiths University of London (2018)
Assayag, G., Bloch, G., Chemilier, M., Cont, A., Dubnov, S.: OMax brothers: a dynamic topology of agents for improvization learning. In: Proceedings of the 1st ACM Workshop on Audio and Music Computing Multimedia (2006)
Ballet, G., Borghesi, R., Hoffmann, P., Lévy, F.: Studio online 3.0: an internet “killer application” for remote access to IRCAM sounds and processing tools. In: Journées d’Informatique Musicale (JIM) (1999)
Bevilacqua, F., Zamborlin, B., Sypniewski, A., Schnell, N., Guédy, F., Rasamimanana, N.: Continuous realtime gesture following and recognition. In: Kopp, S., Wachsmuth, I. (eds.) GW 2009. LNCS (LNAI), vol. 5934, pp. 73–84. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12553-9_7
Boden, M.A.: Computer models of creativity. AI Mag. 30(3), 23 (2009)
Briot, J.-P., Hadjeres, G., Pachet, F.: Deep learning techniques for music generation-a survey. arXiv preprint arXiv:1709.01620 (2017)
Cage, J.: Experimental music. In: Silence: Lectures and Writings, vol. 7, p. 12 (1961)
Chowning, J.M.: The synthesis of complex audio spectra by means of frequency modulation. J. Audio Eng. Soc. 21(7), 526–534 (1973)
Esling, P., Chemla-Romeu-Santos, A., Bitton, A.: Bridging audio analysis, perception and synthesis with perceptually-regularized variational timbre spaces. DAFx2018 (2018)
Fiebrink, R., Caramiaux, B., Dean, R., McLean, A.: The Machine Learning Algorithm as Creative Musical Tool. Oxford University Press, Oxford (2016)
Ghisi, D.: Music across music: towards a corpus-based, interactive computer-aided composition. Doctoral dissertation, Paris 6 (2017)
Hamel, P., Eck, D.: Learning features from music audio with deep belief networks. In: 11th International Society for Music Information Retrieval Conference (2010)
Kingma, D., Welling, M.: Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013)
Kronland-Martinet, R.: The wavelet transform for analysis, synthesis, and processing of speech and music sounds. Comput. Music J. 12(4), 11–20 (1988)
Meredith, D. (ed.): Computational Music Analysis, vol. 62. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-25931-4
Montague, S.: John Cage at seventy: an interview. Am. Music 3, 205–216 (1985)
Risset, J.C., Wessel, D.L.: Exploration of timbre by analysis and synthesis. In: The Psychology of Music, pp. 113–169. Academic Press (1999)
Risset, J.-C.: Fifty years of digital sound for music. In: Proceedings of the 4th Sound and Music Computing Conference (SMC) (2007)
Rodet, X., Depalle, P., Poirot, G.: Speech analysis and synthesis methods based on spectral envelopes and voiced/unvoiced functions. In: European Conference on Speech Technology (1987)
Scurto, H., Bevilacqua, F., Caramiaux, B.: Perceiving agent collaborative sonic exploration in interactive reinforcement learning. In: Proceedings of the 15th Sound and Music Computing Conference (SMC) (2018)
Scurto, H., Kerrebroeck, B.V., Caramiaux, B., Bevilacqua, F.: Designing deep reinforcement learning for human parameter exploration. ACM Trans. Comput.-Hum. Interact. (TOCHI) 28(1), 1–35 (2021)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Tanaka, A., Donnarumma, M.: The body as musical instrument. In: The Oxford Handbook of Music and the Body (2018)
Warnell, G., Waytowich, N., Lawhern, V., Stone, P.: Deep TAMER: interactive agent shaping in high-dimensional state spaces. In: Thirty-Second AAAI Conference on Artificial Intelligence, April 2018
Ystad, S., Aramaki, M., Kronland-Martinet, R.: Timbre from sound synthesis and high-level control perspectives. In: Siedenburg, K., Saitis, C., McAdams, S., Popper, A.N., Fay, R.R. (eds.) Timbre: Acoustics, Perception, and Cognition. SHAR, vol. 69, pp. 361–389. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14832-4_13
Acknowledgments
We thank Frédéric Bevilacqua, Philippe Esling, Gérard Assayag, Goffredo Haus, and Bavo Van Kerrebroeck for their broad contributions to scientific modelling.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Scurto, H., Chemla–Romeu-Santos, A. (2021). Machine Learning for Computer Music Multidisciplinary Research: A Practical Case Study. In: Kronland-Martinet, R., Ystad, S., Aramaki, M. (eds) Perception, Representations, Image, Sound, Music. CMMR 2019. Lecture Notes in Computer Science(), vol 12631. Springer, Cham. https://doi.org/10.1007/978-3-030-70210-6_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-70210-6_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-70209-0
Online ISBN: 978-3-030-70210-6
eBook Packages: Computer ScienceComputer Science (R0)