International Journal of Software Innovation Volume 12 • Issue 1 The Internet of Musical Stuff: Towards an Aesthetically Pliable Musical Internet Marcello Messina, Southern Federal University, Russia* https://orcid.org/0000-0002-8822-3342 Ariane de Souza Stolfi, Universidade Federal do Sul da Bahia, Brazil https://orcid.org/0000-0002-2595-8480 Luzilei Aliel, University of São Paulo, Brazil https://orcid.org/0000-0001-6728-6451 Ivan Simurra, Federal University of Acre, Brazil Damián Keller, Federal University of Acre, Brazil https://orcid.org/0000-0002-0866-3066 ABSTRACT A recent initiative within ubimus research contemplates the development of an internet of musical stuff (IoMuSt) as a concept that interacts with and expands the pre-existing rubric of the internet of musical things (IoMusT). Opposed to the ontological fixedness of things, stuff is pliable, fairly amorphous, changeable depending on usage, context-reliant, either persistent or volatile. It encompasses adaptable and flexible temporalities, featuring non-allotable, non-monetisable and non-reifiable resources. Furthermore, IoMuSt highlights the distinction between object and subject, blurring this crisp separation. The IoMuSt rubric is sustained by aesthetic pliability, fostering an expansion of creative practices and a critical stance towards utilitarian human-computer interaction perspectives. The authors discuss key dimensions of aesthetic pliability as related to flexible infrastructures, open sources and methods, enhanced collaboration and a low ecological footprint. The properties of aesthetic pliability are explored within the realm of two case studies. KeywoRdS Aesthetic Pliability, Comprovisation, Live Patching, Musical Stuff, Open Repository 1. INTRodUCTIoN The usage of the internet as a musical resource is not new. Pre-internet activities can be traced back to the late 1970s, with various artists located in the US proposing musical activities supported by ad hoc networks (Bischoff et al. 1978). During the 1990s, Rich Gold used early versions of the internet within the context of his work in ubiquitous computing at Xerox Parc. Some years later, Chris Chafe DOI: 10.4018/IJSI.344018 *Corresponding Author This article published as an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) which permits unrestricted use, distribution, and production in any medium, provided the author of the original work and original publication source are properly credited. 1 International Journal of Software Innovation Volume 12 • Issue 1 and a team of researchers at Stanford University deployed fully blown synchronous eight-channel sound between two halls located on campus. These early experiences were all characterised by custom software and hardware tailored to handle the limitations of the extant know-how and equipment, bypassing the lack of standard protocols. Despite the technical advances of the following two decades, the usage of the internet as a platform for music-making remained a fairly restricted phenomenon mostly adopted by the artistic and research institutions that had sufficient funding and access to dedicated infrastructure such as, for instance, the Internet 2.0. This situation changed in 2020. Keller, Costalonga and Messina (2020: 11) describe a change of mindset regarding the use of technological support for artistic practice, listing physical isolation and reduced mobility as propaedeutic for an overcoming of acoustic-instrumental practices in favour of domestic, asynchronous and multimodal musical exchanges. Thus, the pandemic context set the stage for the expansive exploration of internet platforms for artistic goals. Obviously, this presented both challenges and opportunities regarding the profile of the musical stakeholders.1 Places that were previously deemed unworthy of artistic exploration suddenly became valid settings for musical endeavours. A panoply of practices based on networked platforms mushroomed, highlighting the need for practice-oriented research not only targeting the available commercial tools but also considering the specific needs of participants previously excluded as doers. Ubiquitous music (ubimus) comprises a body of knowledge that contributes toward these goals. A first wave of ubimus proposals (2007-2014) featured multiple threads characterised by an exploratory and diverse set of experiences that yielded useful hints and unveiled promising areas of investigation (Keller et al. 2010; Miletto et al. 2011). Some proposals were oriented toward the development of infrastructure. Others tended to gather empirical evidence by deploying prototypes and assessing their impact on the creative processes. By targeting distributed resources, these experiences underline the limitations of a one-size-fits-all perspective on musical interaction. In fact, a strong corollary of these initiatives is that imposing a pre-established genre – such as chamber instrumental music – to all internet-based musical practices is prone to exclude ways of doing music that do not fit within the 19th-century European instrumental moulds.2 Avoiding the acoustic-instrumental ways of handling musical experiences is among the most difficult targets set by the first-wave ubimus agenda. Acoustic-instrumental concepts are not only central components of the mainstream trends in musical interaction – including “virtuosic performance” and “self-expression” (Wessel and Wright 2002) – they are also core elements of widely adopted working methods, e.g. orchestras, the concert hall and similar hierarchical forms of social organisation for digital music-making. Complementing the critical analysis of hegemonic musicalinteraction methods, second-wave ubimus frameworks give salience to a characteristic of interaction design which may eventually become a guiding principle for infrastructure developments: aesthetic pliability. Aesthetic pliability comprises emerging approaches to interaction and artistic practice. One trend is represented by the expansion of creative practices enabled not only by the convergence of technological platforms (see Pimenta et al. 2014 for a ubimus perspective on this issue), but also as an answer to the passive role of the audience enforced by the acoustic-instrumental mindset. The exploration of distributed resources encouraged by second-wave ubimus practices demands phenomenologically flexible constructs, such as the Internet of Musical Stuff – IoMuSt (Messina et al. 2022a; Messina et al 2022b; Messina et al 2023). Provisionally, musical stuff can be defined as a phenomenologically oriented shuffle of pliable entities that enable distributed creative activities, employing resources of the musical internet. Being a community-oriented entity, musical stuff features emergent relational properties that become accessible through deployment and usage of resources. Contrary to things, which are vulnerable to imposed hegemonic territorialities, reification, objectification and — in the context of global transactions — monetization; stuff is pliable and fairly amorphous, it changes with usage and relies on context to acquire meaning, it may be persistent or volatile depending on the demands of the stakeholders, it supports handling through flexible temporalities, incorporates value through sharing and readily adapts to non-hierarchical artistic 2 International Journal of Software Innovation Volume 12 • Issue 1 procedures. As a key feature of musical stuff, aesthetic pliability enables the coexistence of diverse designs without constraining the technological infrastructure or the creative procedures. Grounded on a solid yet flexible infrastructure, this approach fosters creative exchanges without imposing aesthetic targets or pre-packaged music genres. A second dimension of pliability encompasses open sources and methods which are crucial to avoid the artificial production of scarcity when producing, sharing, modifying and recycling creative resources (cf. Puckette 2004). Collaborative, horizontal and non-hierarchical social interactions may also be paramount to avoid exploitative, consumeristic, mercantilist and master-slave approaches — often described by corporate-assigned labels such as “property”, “ownership”, “market value” or “owning the real thing”.3 Opposing such labels, a low ecological footprint emerges as another relevant dimension of aesthetic pliability, avoiding the energivorous verification technologies needed to ascertain the “realness” of a thing and its “true ownership” status.4 Second-wave ubimus perspectives encourage a critical stance toward utilitarian human-computer interaction, emphasising a close rapport between aesthetics and interactivity (Keller et al. 2015; Löwgren 2009). Löwgren (2009) highlights design features considered marginal by utilitarian frameworks. For instance, he proposes rhythm as a target of study in technological deployments. When applied to the musical field, this term may be interpreted as equivalent to periodic or metrebased temporal patterns.5 We argue that temporalities is a more appropriate label for the phenomena articulated by Löwgren. Temporalities – or patterns of temporal organisation of creative resources – constitute a central component of ubimus activities. Participants not only handle temporalities through sonic resources. Behavioural and multimodal ecologies also feature temporalities (Keller and Lazzarini 2017). Consequently, the adoption of an interaction metaphor may constrain not only the temporal organisation of the musical outcomes, it may also restrict the range of valid creative strategies compromising the research targets. A case in point is the musical genre networked music performance (Mills 2019), which focuses on latency and jitter as critical variables employed to assess the functionality of infrastructure. It is implied that music-making will be carried out synchronously, ideally within a radius of 300 km., enabling delays that do not disrupt synchronicity. The temporalities envisaged by this musical genre are strongly tied to the instrumental chamber-music format. Contrastingly, an aesthetically pliable design would demand a neutral musical perspective, embracing a wide array of solutions that encompass synchronicity, quasi-synchronicity and asynchronous interaction. To sum up, there are at least two approaches to the deployment of musical activities on the internet. One is genre-centric and views music-making as an emulation of acoustic-instrumental chamber music. The other one fosters designs based on pliable resources and makes use of flexible temporalities. The musical examples chosen for this paper exemplify synchronous activities that are resilient to latency and jitter, offering alternative strategies for the organisation of temporalities in group-oriented artistic practice involving distributed resources. Furthermore, other aspects of design gain prominence, indicating renewed challenges for internet-based musical initiatives that skew preinternet views on music-making. Along these lines, by letting the participants deal with time without resorting to genre-specific clutches, ubimus projects encourage more diverse approaches to handling time in artistic contexts, favouring the free organisation of temporalities over the imposition of metre. Furthermore, this approach highlights the emergence of specific techniques to deal with group-based musical activities. For instance, despite relying on explicit verbal exchanges to support collective decision-making, several ubimus ecosystems do not assume the need for “synchronous face-to-face interactions for musically effective support” (Keller, Costalonga and Messina 2020). The remaining sections of the paper are organised as follows. We review several examples of creative tools that incorporate semantics-based metaphors to avoid an acoustic-instrumental focus, as emergent strategies for musical interaction. Then we present two case studies of internet-based ubimus practice. The first one targets collective musical programming by means of a visual language: live patching. This practice engages with issues related to territoriality (encompassing competitive 3 International Journal of Software Innovation Volume 12 • Issue 1 and cooperative behaviours) and it applies flexible temporalities in collective decision-making. Based on an open sonic repository, the second case also targets a collaborative practice mediated by visual and semantic resources. Improvisatory strategies are highlighted and the caveats and possibilities of the use of massive internet-based resources are analysed through the lens of second-wave ubimus frameworks. The last section discusses the outcomes of these artistic deployments with an eye on the development of an aesthetically pliable musical internet. 1.1 Semantic Strategies to Support Creative Usage of the Musical Internet Semantics-based musical interaction has a historical precedent in verbal-notation practices. Verbal scores were widely adopted by the experimental practitioners of the fifties and sixties, including composers such as La Monte Young and performance artists such as George Maciunas and Joseph Beuys. At the time, the adoption of computational tools involved constraining artistic practices to specialised venues – a requirement that went against the grain of the artistic proposals championed by James Tenney and other composers linked to the artistic collective Fluxus (cf. Lazzarini et al. 2023 for technical details). Contrastingly, the current tendencies in creative music making indicate that computational resources are employed at some stage by a significant body of practitioners. As musical training is not straightforward or widespread, semantics-based musical interaction may furnish a way to promote musical knowledge-sharing without demanding intense and costly training. Soumix6 is a music-sharing online platform that incorporates a classic time-based waveform display. It features basic editing functionalities, such as cutting and pasting. Equalization is also supported. As previously implemented by Miletto et al. (2011) in CODES, participants share audio files and communicate using text. CODES is a web-based music creation system that features a variety of strategies to support collective decision-making, including an integrated chat tool. Participants work on music prototypes (shared persistent representations of musical products) that are constructed through the use of four-second sound samples, provided by the system. This limited pool of materials is usually sufficient for first-time participants. As a musical product constructed through successive iterations involving semantic exchanges among stakeholders, the shared music prototype constitutes one of the central concepts adopted by this type of design. Another online commercial mixing service is UJAM Studio,7 which system targets a karaoke-style type of usage. It supports the analysis of monophonic singing recordings and it uses the extracted Figure 1. The Soumix editor interface 4 International Journal of Software Innovation Volume 12 • Issue 1 Figure 2. UJAM Studio interface (a. record voice, b. apply instrumental presets, c. edit formal structure, d. edit audio sources) sonic information to select accompaniment presets. The users choose among a few popular music genres and the system provides pre-edited acoustic-instrumental recordings for the chosen style. To make up for the limitations of this pre-packaged approach, UJAM Studio supports template editing. A chord editor, based on a matrix of chord ciphers, lets users edit their harmonic sequences. Tools for basic audio editing are also provided. An interesting aspect of the UJAM Studio interface is its ability to deal with mid-level structural information directly (figure 2, c. edit content). Users can define blocks of musical data, assigning them a label and a colour. Then they handle the blocks’ temporal distribution through a drag-and-drop mechanism. This graphical approach to musical data manipulation was proposed and investigated by the musical-representation team at IRCAM in the late 1990s, becoming a staple feature of various music systems (Assayag 1998). A different approach is adopted by the track-assistant tool featured in the Neutron sequencer plugin8. On the one hand, the UJAM Studio attempts to increase usability by abstracting musical data: chords are represented as letters and musical sections are represented as labelled and coloured rectangles. On the other hand, rather than enhancing the graphical user interface, the Neutron system parses interaction into two stages. A first stage provides text descriptors that stand for parametric choices. For instance, the settings for dynamics can be low, medium or high (which stand for piano, mezzo forte and forte dynamic markings). Each of the terms is mapped onto a parametric configuration preset. At a second stage, the stakeholders can use a graphical user interface to tweak the values. For asynchronous usage, this mechanism is computationally efficient and fairly intuitive. The downside is that the user does not have immediate sonic feedback while making her choices. Cartwright and Pardo (2013), Stables et al. (2014) and Seetharaman and Pardo (2016) have also applied semantic descriptors to parametric audio processing. The idea is similar to the procedure 5 International Journal of Software Innovation Volume 12 • Issue 1 Figure 3. The Neutron track-assistant interface implemented in the Neutron track-assistant but rather than splitting interaction into two-stages their systems insert the semantic tags directly into the parametric interfaces. Users can recall previously saved tags or they can create their own terms to describe the sonic qualities produced by the parametric manipulations. Examples of this strategy can be seen in Audealize (Cartwright and Pardo 2013) and SAFE (Stables et al. 2014). Audealize (Cartwright and Pardo 2013; Seetharaman and Pardo 2016) features two systems: one for the configuration of equalisation parameters and another for the control of reverberation parameters. The target of this proposal is the implementation of equalisation and reverberation tools handled through perceptually relevant descriptors rather than through the manipulation of the underlying acoustic variables (such as reverberation time, absorption and other low-level parameters). The Audealize interface displays user-created descriptors of parametric configurations. Its display applies the metaphor of the tag cloud.9 Parametric configurations can be changed by clicking on each tag or alternatively by manipulating the control sliders corresponding to the gain at each frequency band of the graphic equaliser or by setting the knobs to control the reverb parameters (in this implementation: reverb time, echo density, clarity, central time and spectral centroid). Rather than implementing a web-based system, Stables and coauthors (2014) chose to enhance the functionality of desktop based digital audio workstations through semantic control. They adopted semantic tags to control equalisation, distortion, compression and reverberation parameters, through the deployment of audio plugins (grouped under the label Semantic Audio Feature Extraction SAFE). The SAFE plugins feature standard controllers and displays (such as knobs and graphic displays of parameters) and provide a way to attach a descriptor to each parametric configuration. Thus, by typing or selecting a word or a sequence of words the users can save and recall their control settings. Complementarily, the tool allows them to furnish metadata information including their age and location, music production experience, their choice of music genre and the instrumentation employed in their mix. An analysis of the collected terms indicated that equalisation and compression may have a common vocabulary, while reverb and distortion feature different descriptive choices (Stables et al. 2016). The compressor terms tend to correlate with gain reduction applied on the signal. Loud, fat and squashed generally refer to extreme compression settings, whereas subtle, gentle and soft tend to describe minor adjustments to the amplitude envelope. Distortion features are usually related to the perceived roughness of the sonic transformation, with terms such as fuzz and harsh clearly separated from subtle, rasp and growl. 6 International Journal of Software Innovation Volume 12 • Issue 1 Figure 4. The Audealize equalisation word-cloud interface, featuring semantic descriptors written in Spanish Figure 5. The SAFE plugins: The box on the right of each plugin lets the users save and recall semantic tags describing the sonic result of the parametric configurations Equalisation descriptors comprise a wide selection of categories. But only the terms that refer to specific regions of spectral energy such as bass, mid and full fall into consistently independent partitions. Reverb terms tend to be based on the size and on the characteristics of the acoustic spaces. For instance, hall and room share similar feature spaces, while terms such as soft, damp and natural belong to a separate group. In contrast with these encouraging results, Seetharaman and Pardo (2014) point out that the terms most commonly used to describe reverberator profiles do not map well onto specific parametric reverberation settings. Hence, they conclude that acoustic settings yield more predictable results than comparable semantic controls when the audio-processing target is reverberation. Summing up, recent proposals in semantics-based interaction for audio processing point to promising avenues of development that need further study. While the existing proposals have focused 7 International Journal of Software Innovation Volume 12 • Issue 1 on validating user-defined tags, it is not clear whether spontaneous terms could be applied across a wide range of applications and contexts of use, particularly when involving stakeholders with little musical expertise. The reviewed studies seem to imply that the target user-base includes just expert musicians with considerable experience in manipulating studio equipment and audio production tools. For this population profile, it may be feasible to replace GUI-based interaction with the usage of technical descriptive terms. But this assumption may not be true for casual users. If, on the one hand, it is reasonable to expect that audio-production stakeholders will invest time to familiarise themselves with a specific tool set, on the other hand it is not clear that a large technically oriented vocabulary would be preferable over handling multiple GUI elements. In user-defined descriptor usage, a very large set of sonic outcomes relying on detailed parametric control needs to be matched to a large set of concepts (ranging from familiar everyday notions to exotic and specialised jargon). For novices, this approach to interaction may become unwieldy. Taking into account these issues, Stasis and coauthors’ (2017) proposal seems to be reasonable: focus just on two frequent equalisation descriptors - warm and bright - and try to untangle the implications of their usage. Interestingly, they found that the warm adjective is 100% reliable (within the constraints of their methods) while the use of bright achieves 90% of accuracy. This performance improvement – when compared to tasks that deal with an unlimited set of descriptors – points toward an alternative strategy to reduce the cognitive toll of timbre manipulations. Rather than displaying a large range of options from the start – featuring dozens of descriptors – a tag selection strategy could apply a process of progressive refinement, starting from broad categories and leaving the subtle differences to decisions at later stages, until very subtle parametric changes become uninteresting or irrelevant. When interpreting these results from a broader creativity-centred perspective, other aspects come to the forefront. Audio-processing tools are rarely used in isolation. They are often employed within the context of a wide range of creative activities (such as selecting sources, audio processing, editing and mixing), configuring what is sometimes called the audio-production chain (Tatlas et al. 2003). Given this context of use, factors such as the time invested and the resources used are as important as the quality of the creative outcomes. Ubimus research methods might help to gauge the impact on the creativity factors of the adopted interaction support strategies (Pimenta et al. 2014). For instance, strategies such as those applied by Audealize and SAFE could be employed in conjunction with mixing tools (for instance, using SoundSphere or Audacity) to assess: the temporal investment on the manipulation of sonic resources, the ratio between resource usage and creative waste and the creative profile of the finished products. Long temporal investments or lack of originality of the creative products may indicate cognitive demands unsuited for the stakeholders’ profiles. The analysis of these variables might help to untangle the impact of the design on the creative activities for the specific population profile. Having discussed representative proposals of semantics-based strategies employed to handle musical resources, we will now turn our attention to two ubimus studies that apply semantics-based approaches to enable artistic practice based on networked resources. We chose two well-documented prototypes to provide a firm basis for our analysis, emphasising the creative outcomes. We acknowledge that the creative results are not usually needed in interaction-design projects, since the byproducts of the activity are not central elements of the design cycle. Nevertheless, our focus on artistic practice forces us to consider the material outcomes as complementary evidence of support for creative action. 2. CASe oNe: LIVe PATCHING The term “live patching” derives from the concept of live coding, defined by Thor Magnusson as a “form of musical performance that involves the real-time composition of music by means of writing code” (Magnusson, 2011). While live coding may or may not involve the use of text-based user interfaces, live patching exclusively targets graphical programming languages. The smoother learning curve granted by visually oriented interaction suits the collaboration and educational purposes of 8 International Journal of Software Innovation Volume 12 • Issue 1 Table 1. Summary of the characteristics of the reviewed systems Tool Context of use Architecture Target Interaction features UJAM Studio Online fixed karaoke, instrumental accompaniment chords represented as letters; musical data represented as labelled, coloured blocks Neutron track assistant Neutron sequencer plugin fixed choice of factory presets for musical data sequences two-tier approach: presets chosen through successive multiple-choice questions; standard parametric GUI controls SAFE DAW plugin DAW plugin equalisation, reverberation, compression, distortion operators represented as user-defined descriptors; standard parametric GUI controls Audealize Online fixed equalisation, reverberation operators represented as user-defined descriptors; word cloud display of operators; community-oriented design second-wave ubimus initiatives. Despite this advantage, when involving participants with no previous programming experience, live patching demands strategies to facilitate the learning process of patching languages. 2.1 Characteristics of Kiwi Kiwi is a graphic audio patching tool in line with software such as Pure Data or Max, with an added collaborative dimension that allows users to work together in a network on the same processing or on the same digital audio composition (Paris et al., 2017). Kiwi’s model focuses on sharing a workspace, i.e. the patch canvas, while control parameters and audio signals are saved locally. An important feature of Kiwi is the presence of rooms where participants can create collaborative patches and make them available to other network members. All users connected to the server have access to the list of rooms: from this window they can download, delete, upload, rename, duplicate patches. Importantly, each room allows multiple users to connect simultaneously, allowing changes to the shared patch in real time. All participants can view and edit all documents that users create online (Paris et al., 2017). A collaboration between the Federal Universities of Acre and Paraíba (in Brazil) and the Paris 8 University (France), inscribed within the Live/Acc/Patch project (UFAC/CNPq) introduced the possibility of using Kiwi as a tool to alleviate the difficulties of collaborating over large distances (Messina et al. 2019).. 2.2 Simultaneous Intercontinental Live Patching After a series of local practices intended to familiarise quickly with Kiwi as a patching environment, towards the end of 2018, the members of the Live/Acc/Patch project started live patching between the two Brazilian states of Acre and Paraíba. In April 2019, three attempts at intercontinental live patching between France and Brazil were made. A first set of interstate sessions occurred between Acre and Paraíba in the last months of 2018. Sessions are available on the Kiwi server as LIVE-ACC-PATCH UFAC/UFPB followed by consecutive numbers. Four sessions were deployed between October 17th and November 28th. 2.3 The Practical and Conceptual Limits of Simultaneous Remote Interaction The most appealing feature offered by Kiwi is, by far, the support for simultaneous and remote collective interaction through patching (Paris et al,2017). Some problems occurred upon the first intercontinental test on April 11th, triggering a broader reflection on the desirability of simultaneity 9 International Journal of Software Innovation Volume 12 • Issue 1 in the context of remote interaction. Kiwi allows for simultaneity to be temporally displaced. Patcher rooms created several months ago remain available for access and modification by users. Kiwi stores collective patchers and never concludes them. On the network, they remain open for the community to continue modifying them. Thus, collective events involving different places could also be temporally disphased. This temporal handling of the collective patching activity is precisely in line with our aforementioned reflections on ubimus temporalities. Another important conceptual node is remoteness, which could be described as the simulation of a presence in loco that manifests itself via patching operations on the computer screen. Regarding systems of biometric detection, Joseph Pugliese the expression “metaphysics of presence” borrows from Derrida and identifies a set of potential failures involved in recognising and passing on the indicators of human presence (Pugliese 2014). Telepresence, defined as early as the 1990s as “the perception of presence within a physically remote or simulated site” (Draper, Kaber and Usher 1998),10 is in fact one of the most thoroughly discussed concepts of early computer-based artistic practice. During the various live patching sessions, project members found themselves relying on textual comments on the patching canvas in order to make up for the absence of verbal communication with distant participants. The written text, in this case, operated to bridge a gap in the interaction with the other stakeholders. 3. CASe Two: PLAySoUNd.SPACe The open online tool Playsound.space (Stolfi, Milo and Barthet 2019) aims to increase the circulation and transformation of sonic resources through flexible temporalities, by overlapping several sound layers, and by promoting collaborative participation. The tool extracts accessible and modifiable Creative Commons licences sound content and targets free improvisation practices, allowing a free layering and looping of the sound samples and free time transformations, in contrast to the timeline grids typical of DAWs. Playsound.space employs the Freesound.org API (Akkermans et al. 2011) to access their database of sound samples (more than 200.000 items from Creative Commons), browsable through semantics-based queries. The search mechanism is implemented using Angular.js two-way data binding, hence sounds are retrieved while the user is typing the words without the need to press enter. Selected sounds are united as a section with each collection featuring a unique URL that can be saved for later access. A key feature of this software is the use of spectrograms as visual support for queries. This enables the participant to conduct searches while performing, as the spectrograms can show visual information about the timbre, shape, texture, envelope or harmonic content of the sound samples, with no need to listen to them. This iconic information supports fast decision-making, especially in improvisational contexts, without requiring specific knowledge of sound names or details of the sonic contents. Openness is the main guiding principle of the software’s design, as playsound.space is fully accessible registration/login-free and with no tracking of user data. Simplicity is another main principle, as the software functions with low server resources and no database, once the sounds are retrieved from Freesound through its API and all sound processing happens locally — Playsound could even function without the internet, as the sounds remain in the browser cache while the page is still open. The player loops and plays samples and permits some basic editing, such as controlling loop start/endpoint, pan, and time-based stretching; it also features a master volume control and a record button that saves the performance as a WAV file. The loop-based process supported by Playsound is tailored for free-improvisation usage: it allows creating polyrhythmic and cumulative sonic textures of variable durations, not restricted to metre-based patterns. Two case studies involving Playsound are discussed: Noise Symphony and Tender Buttons. 10 International Journal of Software Innovation Volume 12 • Issue 1 Figure 6. Noise Symphony X (Screenshot by the author) 3.1 Noise Symphony The Noise symphony series (Stolfi 2021) is based on a query for the term “noise” that retrieved around 9000 search results. Using the spectrograms as an aid for sound selection, 400 sound samples were collected for these improvisation-based pieces.11 The selection strategy yielded a varied collection of sounds, and the artist selected mostly short sounds that afforded multiple possibilities of composition and juxtaposition, emphasising polyrhythmic possibilities.12 Long sounds were avoided to save processing power. Methods. The artist played this collection of sound samples together with voice-based free improvisation, exploring also extended vocal techniques. The videos were composed by layering the software’s interface and a webcam showing the artist’s face while singing (figure 6). Results. In a comprovisation context during the Covid-19 pandemic (Aliel et al. 2018), this collection of sounds furnished a basic structure of different online live sessions online. The work was presented live in different conferences: Seminário UFSB em Rede, Web Audio Conference, The Ubimus Symposium and the 18 ¿Música? Festival, at the University of São Paulo (2023). A video edition, Noise symphony X was also featured at the ubimus event organised by CLB-Berlin in 2022 and at Festival Música Nova Gilberto Mendes (2019 and 2020). 3.2 Tender Buttons Tender buttons | Sound | space13 was premiered at the 2018 Web Audio conference. Ariane Stolfi and Alessia Milo performed together using Gertrude Stein’s poem Tender Buttons. They selected this poem because it is a significant work made by a writress, it is free to use and describes a lot of objects furnishing a wide selection of words to search for. Part 3 of the poem Rooms was used in this performance. Methods. Firstly, Ariane’s voice was recorded reading the poem. The recordings were uploaded to the Freesound Library.14 The current version can’t provide a multiplayer experience but features a chat interface that can be used to share words to be queried by the software. The tender-buttons keys were later accessed by Playsound’s anonymous chat, where words are clickable to trigger search queries. Both computers were connected to a mixer and to an amplified sound system. The audience was watching Ariane’s Screen on the projector. During the 21-minute performance, Alessia retrieved 11 International Journal of Software Innovation Volume 12 • Issue 1 Figure 7. Tender Buttons | Sound | Space screen, showing Playsound chat interface (Source: https://www.youtube.com/watch?v=LiNb_T8oluA) pre-recorded readings from Freesound using the selected keyword, playing them while copying and pasting the poem quotations on the chat interface (figure X). At the same time, Ariane selected words from the pasted text on the chat window to perform live semantic queries. This process yielded an improvised live soundtrack based on words from the poem. Results. The result showcased the poem’s reading and a noisy layer of variegated sonic loops. An interesting aspect of this performance is that a lot of recorded samples produced by the authors were included. Playsound retrieves only public domain or free to use sound samples from Freesound library. This restriction avoids possible legal issues and also encourages the publication of freely accessible materials. To use these materials in Playsound, they had to publish them as creative commons with a free to use licence on Freesound. 3.3 Implications for developing an Aesthetically Pliable Musical Internet An open issue in ubiquitous music research is how to approach diverse musical traditions and cultural contexts without subscribing to the prescriptive or culture-cleansing tendencies that plagued 20thcentury music theory. Keller, Costalonga and Messina (2020: 3) ask whether musical robots, musical algorithms or refined methods of data analysis may eventually replace the acoustic-instrumental ways of music-making. As part of the answer, we see two directions in recent developments that may shape the future expansion of the musical internet: a. There is an increased awareness of the impact of legacy concepts and technology on emergent internet-based practices (Lazzarini et al. 2023; Karkina et al. 2023; Messina et al. 2022b; Zawacki and Johann 2014); b. There is a tendency to incorporate new resources while promoting adaptations of extant frameworks. These two tendencies have been described as the “push and pull” forces that shape music infrastructures, including networked resources. The tensions between the established musical genres and the emergent creative practices become more acute as new research threads engage with networked assets. The second case-study presented in this paper illustrates these issues. For instance, consider the ratio between the number of samples effectively employed and the sonic sources potentially applicable to Noise Symphony (4 out of 90). The process of selecting the sounds requires getting acquainted with a pool of 9000 samples. Let us say we limit the soundbites to short durations, averaging 10 seconds. To listen once to this material 12 International Journal of Software Innovation Volume 12 • Issue 1 demands a nonstop session lasting a minimum of 25 hours. This task, of course, becomes unwieldy if assigned to a single practitioner in synchronous mode. Consequently, other approaches are necessary when dealing with resources that exceed the limits of individual decision-making.15 Let us label this problem, large-scale creative-resource allocation. Barriers to large-scale resource allocation can be defined as temporary or permanent breaks in the supply chain of creative resources, caused by the stakeholders’ limitations or by the lack of access to material support. Consequently, a top-down approach to the design of the musical internet will encounter limitations prompted by at least two factors: the diverse profile of the stakeholders and the wide variability in local conditions. These observations reinforce the need for a serious investment in diversified strategies for support of internet-based musical activities. In the context of ubimus activities, when the ability of the stakeholders to sift through the available materials is exceeded, caveats in creative-resource allocation may arise. The causes are varied. One cause is the need to keep the aesthetic choices open. Raw materials may originate in the local environment (e.g., time tagging – Keller et al. 2010), they may be available through distributed platforms (case 2), they may come from other stakeholders (as in live patching – case 1) or they may be generated algorithmically. Playsound.space supports the use of word-tokens to handle an initial selection of resources. An advantage of this approach is the possibility to apply automatic translations. Thus, the tokens become accessible beyond English speakers, as exemplified in Audealize and Playsound.space. In some cases, multilingual support is not optional. This is both a limitation and an opportunity for designs that engage with cross-cultural dimensions. Unexpected outcomes from translation processes can also be explored for creative purposes. This technique is employed by Aliel et al. (2018). Generative tools produce raw text which is then translated to languages not spoken by the participants. This textual material is rendered as sound by means of text-to-speech techniques. Thus, the translation process is reconfigured as a sound-synthesis strategy. Within the thread of semantic-based ubimus initiatives, a proposal that has gained weight during the last few years is Creative Semantic Anchoring (ASC, from its original Portuguese formulation Ancoragem Semântica Criativa), which is based on the use of verbal indicators to enhance and facilitate creative processes. A first ubimus experience involved the deployment of a set of methodological tools called Interpretative Tokens (in Portuguese, Fichas Interpretativas) to advance the study of semantic elements associated to extended techniques with special attention to the work by Flausino Valle (Keller and Feichas 2018; Keller et al 2020; Messina, Feichas and Ribeiro, 2021). Derived ubimus interventions have turned to the live-coding tool Orca (Messina et al., 2021), documented as a series of original comprovisational pieces (Messina and Aliel, 2019; Messina and Mejía, 2020). Simurra et al. (2023) introduced the concept of Radical ASC. By considering the performativity of semantics in shaping the musical and epimusical characteristics of the creative processes, ASC transcends “traditionally notated” instructions. Furthermore, radical forms of ASC may occur when the agents facilitate the production of creative outcomes whereby the linguistic sign and its meaning constitute the most important or sometimes the only pertinent musical parameter. In this sense, Radical ASC transcends the usage of verbal tokens as creative facilitators: verbal enunciations may be in fact turned into actual musical material. Beside accessing large amounts of resources, another scenario that imposes barriers to the allocation of distributed materials involves the reduction of the stakeholders’ ability to make informed choices. This caveat may be due to temporary breaks in internet connectivity or it may be due to a stakeholder’s condition that precludes the practitioner from gaining full access to musical information. Temporary caveats have been handled by ubimus deployments in various non-standard settings, pointing to a rapport between casual interaction and everyday creativity (Keller 2020). Some challenges have been addressed by means of assistive technology. Various assistive techniques may be adapted to align accessibility with artistic pliability. This accessibility-oriented research points to the need to move beyond instrument-centric views toward the development of adaptive ubimus ecologies (Chakraborty et al. 2022). The logic applied to learning is inverted. Instead of forcing the stakeholders 13 International Journal of Software Innovation Volume 12 • Issue 1 to modify their behaviours to fulfil the requirements of “the instrument”, adaptive ubimus designs envisage functionalities shaped by the behavioural patterns of the subjects. Thus, the attributes of this type of support depend on the specific context and profile of the participants. In this case, musical know-how is captured through actions rather than imposed as a ready-made genre-based solution. Aesthetic pliability emerges as a central design target of second-wave ubimus frameworks, highlighting the potential of interaction aesthetics as a conceptual tool for the expansion of the musical internet and pointing to resources furnished by the semantic web. Interaction frameworks include ASC, Radical ASC, multilingual techniques and explorations of the transitions between word and sound as resources for creative action. Both of the studies presented in this paper incorporate semantics-based strategies featuring specific contributions. Live-patching involves the usage of a visual-programming tool, thus it combines source code with visually oriented techniques to handle code objects. A common working space, or canvas, is defined as a “patch” providing a territorial metaphor for shared creative action. As observed in the sessions, the stakeholders both compete and collaborate to attain their objectives in a process that demands intense verbal exchanges, in this case enabled by repurposing functionalities of the tool. These behaviours are aligned with the strategies supported by the ASC framework which encourage the use of verbal resources as a way to ground collective decisions while establishing a shared knowledge-base. Similarly, both of the artistic projects realised with Playsound.space involve the use of semantic tokens as a first stage in selecting sound samples. This strategy is also complemented with a visually oriented technique, in this case involving the use of sonograms that enable a quick glance at sonic contents. The stringent demands of quasi-synchronous improvisatory practice indicate that a multitiered approach may be a path to handling resource-selection when the pool is large. A caveat of this technique is the computational load of analysing the spectral content of every sample. This option may not be feasible on devices with limited RAM or when the spectral information needs to be shared among many stakeholders. There are several computational methods that can be applied to simplify this task. But to make an informed decision regarding the optional designs, the impact of reduced visual information on decisions based on spectral representations needs to be untangled. This is an area of ongoing research. The creative processes observed in the studies discussed above point to caveats in two areas of support for music-making involving distributed resources. One aspect is knowledge-sharing. To reach a consensual decision, all stakeholders need to have access to timely and relevant information of the targeted outcome including not just the shared materials but also the motivations behind the proposed actions (Miletto et al. 2011). This implies a continuum of methods ranging from sharing minimal information linked to each proposal to sharing all the information produced while interacting with the local resources. Both extremes seem problematic. Relevant and detailed information is a good tool to constrain choices and to point to potential relational properties that may be explored throughout the creative cycle (Keller et al. 2014). When faced with the problem of what to exclude from representations, designs that delay the choices to later stages of the creative activity seem to fare better than those that start from readymade packages.16 In the case of live patching, sharing complex modules of visual code during the early stages tends to reduce the ability of the partners to act on the shared patch because of the increased knowledge necessary to grasp the possibilities of the proposal. The solution found by the participants was to give priority to small functional units that eventually composed larger modules. Complementarily, the Playsound.space examples indicate that large amounts of materials tend to be ignored unless a significant effort is invested in the preparation and documentation of resources (as exemplified in Tender Buttons). This caveat indicates an opportunity to delegate some choices to automated processes. For instance, rather than interacting directly with the contents of the sound repositories the participants may control automated decision-making processes. This might streamline the creative-action metaphor. If more detailed selection is needed, the design may fall back to the manual-selection approach. Another aspect is the respect for and the expansion of the musical internet as a communityoriented common good. This issue cannot be overstated. It was at the centre of the first-wave initiatives 14 International Journal of Software Innovation Volume 12 • Issue 1 (Pimenta et al. 2014) and given the renewed corporate attempts to monetize all internet assets, it is now a most tangible concern of second-wave ubimus approaches (cf. Messina et al. 2023). The two musical studies described above furnish partial solutions for this conundrum. The active engagement with open repositories championed by the Playsound.space project is exemplary in its balance between achieving personal artistic goals and moving forward the agenda toward a freely accessible and community-oriented musical internet. This is perfectly aligned with the ubimus vision of removing barriers to music-making. The live-patching initiative complements this approach by highlighting the use of volatile resources as potential assets as a way to expand our understanding of the musical internet. This is a core aspect of the research on musical stuff. As discussed by Messina et al. (2022b), the concept of musical thing does not encompass the diversified musical experiences that characterise ubimus frameworks. Live patching provides a good example of the difference between things and stuff. The volatile resources that are the core of live patching can hardly be typified as “things”: They are not persistent; The objective of a participant when sharing a functional programming unit may not entail fixing its functionality or delimiting a territory of ownership. This attitude tends to undermine the groups’ objective of arriving at a consensual creative outcome. Visual code is exchanged as a way to construct shared functional modules that partially fulfil the musical expectations of most partners. Thus, it is implied that the exchanges are temporary and that a significant amount of shared resources are eventually discarded. This modus operandi underlines the relevance of the creative residuals or waste as a major parameter to consider when assessing the impact of a creative-action metaphor. Stuff entails recycling and circulation of transformed matter, rather than the fixed reification enforced by the musical thing. Thus, two properties of musical stuff may help to further the ubimus agenda of a musical internet as “a phenomenologically oriented shuffle of pliable entities that enable distributed creative activities”. One property is the inclusion of volatility as a design quality, highlighting the ongoing cycle of transformation between resources, products and waste with both potentially beneficial and detrimental impacts on ubimus ecosystems. A case in point is the enhancement of the sustainability of open resources for creative practice by providing mechanisms to discard low-quality materials and by introducing recycling techniques. Another property involves the reduction of rivalry as a driving force for the expansion of the musical internet. Rather than promoting competition among stakeholders, stuff-oriented designs target non-rivalry and anti-rivalry as the two properties to be supported by infrastructure developments. How to achieve this goal is a complex problem that demands weighing local cultural demands and expanded standards while encouraging a respectful attitude toward the common goods. CoNFLICTS oF INTeReST We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome. FUNdING STATeMeNT No funding was received for this work. CoRReSPoNdING AUTHoR Correspondence should be addressed to Marcello Messina (messina@sfedu.ru) PRoCeSS dATeS Received: 12/19/2023, Revision: 1/24/2024, Accepted: 1/24/2024 15 International Journal of Software Innovation Volume 12 • Issue 1 ReFeReNCeS Akkermans, V., Font Corbera, F., Funollet, J., De Jong, B., Roma Trepat, G., Togias, S., & Serra, X. (2011). Freesound 2: An improved platform for sharing audio clips. In A Klapuri, & C Leider, Editors. Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR 2011). Miami, FL: University of Miami. Aliel, L., Keller, D., & Costa, R. (2018). The Maxwell Demon: A proposal for modeling in ecological synthesis in art practices. Música Hodie, 18(1), 103–116. doi:10.5216/mh.v18i1.53575 Assayag, G. (1998). Composition assistée par ordinateur: nouveaux chemins de création. Ircam. Bischoff, J., Gold, R., & Horton, J. (1978). Music for an interactive network of microcomputers. Computer Music Journal, 2(3), 24–29. doi:10.2307/3679453 Cartwright, M., & Pardo, B. (2013), Social-eq: Crowdsourcing an equalization descriptor map. In Proceedings of the International Society for Music Information Retrieval (ISMIR 2013)(pp. 395-400). Curitiba, PR: ISMIR. Chakraborty, S., Yaseen, A., Timoney, J., Lazzarini, V., & Keller, D. (2022). Adaptive touchless whole-body interaction for casual ubiquitous musical activities. In Proceedings of the International Computer Music Conference (ICMC2022) (pp. 132-138). Limerick, Ireland: University of Limerick Dack, J. (2001). At the limits of Schaeffer’s TARTYP. CD Rom Nowalls Conference Proceedings, (pp. 1-12). Research Gate. Draper, J. V., Kaber, D. B., & Usher, J. M. (1998). Telepresence. Human Factors, 40(3), 354–375. doi:10.1518/001872098779591386 PMID:9849099 Harazim, M., & Comment, T. (2017). Music, Computers and Culture in Live Coding. [Bmus Dissertation, University of Aberdeen, Scotland]. Karkina, S., Mena, J., Valeeva, R., Yarmakeev, I., Dyganova, E., & Bhullar, M. (2023). ‘Fostering future music teachers’ professional skills: Developing a signature pedagogy using e-learning. Frontiers in Education, 8, 8. doi:10.3389/feduc.2023.1162748 Keller, D. (2020). Everyday musical creativity. In Ubiquitous Music Ecologies (pp. 23–51). Routledge. doi:10.4324/9780429281440-2 Keller, D., Barreiro, D. L., Queiroz, M., & Pimenta, M. S. (2010). Anchoring in ubiquitous musical activities. Proceedings of the International Computer Music Conference (ICMC 2010) (pp. 319-326). MPublishing, University of Michigan Library. Keller, D., Costalonga, L., & Messina, M. (2020). Editorial: Ubiquitous Music Making in COVID-19 Times. Proceedings of The Workshop On Ubiquitous Music (UbiMus 2020) (pp. 3-16). Ubiquitous Music Group. Keller, D., & Feichas, L. (2018). Ecocompositional and performative strategies for creative usage of everyday sounds: Creative semantic anchoring. Leonardo, 51(2), 195–196. doi:10.1162/LEON_a_01526 Keller, D., & Lazzarini, V. (2017). Ecologically grounded creative practices in ubiquitous music. Organised Sound, 22(1), 61–72. doi:10.1017/S1355771816000340 Keller, D., Messina, M., da Silva, C. E., & Feichas, L. V. (2020). Embasamento da Ancoragem Semântica Criativa: Estudo Exploratório com Emulações Instrumentais. Journal of Digital Media & Interaction, 3(5), 117–132. Keller, D., Otero, N., Lazzarini, V., Pimenta, M. S., de Lima, M. H., Johann, M., & Costalonga, L. (2015). Interaction aesthetics and ubiquitous music. Creativity in the digital age, 91-105. Keller, D., Otero, N., Lazzarini, V., Pimenta, M. S., Lima, M. H., Johann, M., & Costalonga, L. (2014). Relational properties in interaction aesthetics: The ubiquitous music turn. In Proceedings of the Electronic Visualisation and the Arts Conference (EVA 2014). London: BCS, Computer Arts Society Specialist Group, London. Lazzarini, V., Keller, D., & Radivojevic, N. (2023). Issues of ubiquitous music archaeology: Shared knowledge, simulation, terseness, and ambiguity in early computer music. Frontiers in Signal Processing, 3, 1132672. doi:10.3389/frsip.2023.1132672 16 International Journal of Software Innovation Volume 12 • Issue 1 Lima, M. H., Keller, D., Pimenta, M. S., Lazzarini, V., & Miletto, E. M. (2012). Creativity-centred design for ubiquitous musical activities: Two case studies. Journal of Music. Technology and Education, 5(2), 195–222. Löwgren, J. (2009). Toward an articulation of interaction esthetics. New Review of Hypermedia and Multimedia, 15(2), 129–146. doi:10.1080/13614560903117822 Magnusson, T. (2011). Algorithms as scores: Coding live music. Leonardo Music Journal, 21, 19–23. doi:10.1162/ LMJ_a_00056 Messina, M., & Aliel, L. (2019, October). Ubiquitous music, Gelassenheit and the metaphysics of presence: Hijacking the live score piece Ntrallazzu 4. In 14th International Symposium on Computer Music Multidisciplinary Research (pp. 685-695). Research Gate. Messina, M., Aliel, L., Mejía, C. M. G., Célio Filho, M., & de Melo, M. T. S. (2021). Live Coding on Orca, the Geopolitics of the English Language and the Limits of Creative Semantic Anchoring: A Preliminary Hypothesis. In Proceedings of the 11th Workshop on Ubiquitous Music (UbiMus 2021) (pp. 57-61). Porto, Portugal. Messina, M., Célio Filho, M., Mejía, C. M. G., Keller, D., Aliel, L., & Simurra, I. (2022a). A Internet do Bagulho Musical (Internet of Musical Stuff)-IoMuSt. In Proceedings of the Ubiquitous Music Symposium (ubimus2022) (pp. 85-93). Curitiba, PR: G-ubimus. Messina, M., Feichas, L. V., & Ribeiro, L. P. (2021). Musique Concrète Instrumentale and Coloniality of Knowledge: Helmut Lachenmann, Flausino Valle and the Euro-Normative Bias of New Music Genealogies. In En, desde y hacia las Américas: músicas y migraciones transoceánicas (pp. 653–668). Dykinson. doi:10.2307/j. ctv282jfwp.41 Messina, M., Keller, D., Aliel, L., Gomez, C., Célio Filho, M., & Simurra, I. (2022b). The Internet of Musical Stuff (IoMuSt): Ubimus perspectives on artificial scarcity, virtual communities and (de) objectification. INSAM Journal of Contemporary Music. Art and Technology, 9, 26–50. Messina, M., & Mejía, C. M. G. (2020). Contracapas for remote double bass and effects: Creative Semantic Anchoring, Corpus Linguistics and Remote Interaction. In Anais do X Workshop de Música Ubíqua (UbiMus 2020) (pp. 173-174). Messina, M., Mejía, C. M. G., Simurra, I., Keller, D., Célio Filho, M., Aliel, L. (2023). Ubiquitous Music, Untokenizable Resources and Creative-Action Metaphors: Introducing the Internet of Musical Stuff (IoMuSt). Unpacking Creativity: Culture, Innovation, and Motivation in Global Contexts, 128-152. Messina, M., Svidzinski, J., de Menezes Bezerra, D., & Ferreira, D. (2019). Live patching and remote interaction: A practice-based, intercontinental approach to kiwi. In 14th International Symposium on Computer Music Multidisciplinary Research (p. 696-703). IEEE. Miletto, E., Pimenta, M., Bouchet, F., Sansonnet, J., & Keller, D. (2011). Principles for music creation by novices in networked music environments. Journal of New Music Research, 40(3), 205–216. doi:10.1080/092 98215.2011.603832 Mills, R. (2019). Telematics, art and the evolution of networked music performance. Tele-Improvisation: Intercultural Interaction in the Online Global Music Jam Session, (pp. 21-57). Springer. doi:10.1007/978-3319-71039-6_2 Minsky, M. (1980). Telepresence. Omni, New York, 1980, 45–51. Paris, E., Millot, J., Guillot, P., Bonardi, A. & Sèdes, A. (2017). Kiwi: vers un environnement de création musicale temps réel collaboratif premiers livrables du projet Musicoll. Journées D’Informatique Musicale. Pimenta, M., Keller, D., Flores, L., Lima, M., & Lazzarini, V. (2014). Methods in creativity-centred design for ubiquitous musical activities. Ubiquitous Music. doi:10.1007/978-3-319-11152-0_2 Puckette, M. (2004). Who owns our software? A Firstperson Case Study. In Proceedings of the International Symposium on Electronic Art (pp. 200-202). IEEE. Pugliese, J. (2013). The alleged liveness of “Live”: Legal visuality, biometric liveness testing and the metaphysics of presence. Law, Culture And. Visual Studies, 649–669. 17 International Journal of Software Innovation Volume 12 • Issue 1 Rivlin, L., Ittelson, W. & Proshansky, H. (1976). Environmental psychology: people and their physical settings. Holt, Rinehart. Seetharaman, P., & Pardo, B. (2014). Reverbalize: a crowdsourced reverberation controller. Proceedings of the 22nd ACM International Conference On Multimedia. ACM. doi:10.1145/2647868.2654876 Seetharaman, P., & Pardo, B. (2016). Audealize: Crowdsourced audio production tools. Journal of the Audio Engineering Society, 64(9), 683–695. doi:10.17743/jaes.2016.0037 Simurra, I., Messina, M., Aliel, L., & Keller, D. (2023). Radical Creative Semantic Anchoring: Creative-action metaphors and timbral interaction. Organised Sound, Volume 28, Issue 1. Socially Engaged Sound Practices, 28(Part 2), 64–77. doi:10.1017/S1355771822000322 Stables, R., De Man, B., Enderby, S., Reiss, J., Fazekas, G., & Wilmering, T. (2016). Semantic description of timbral transformations in music production. Proceedings of the 24th ACM International Conference On Multimedia (pp. 337-341). Amsterdam: ACM. doi:10.1145/2964284.2967238 Stables, R., Enderby, S., Man, B. D., Fazekas, G., & Reiss, J. D. (2014). SAFE: A system for the extraction and retrieval of semantic audio descriptors. In Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 2014). Taipei, Taiwan: ISMIR. Stasis, S., Hockman, J., & Stables, R. (2017). Navigating descriptive sub-representations of musical timbre. In Proceedings of the Conference on New Instruments for Musical Expression (NIME 2017) (pp. 56-61). Copenhagen, Denmark: Aalborg University. Stolfi, A. (2021). Noise Symphony. Proceedings of the Web Audio Conference. https://webaudioconf.com/_data/ papers/pdf/2021/2021_23.pdf Stolfi, A., Milo, A., & Barthet, M. (2019). Playsound. space: Improvising in the browser with semantic sound objects. Journal of New Music Research, 48(4), 366–384. doi:10.1080/09298215.2019.1649433 Tatlas, N., Floros, A., Hatziantoniou, P., & Mourjopoulos, J. (2003). Towards the all-digital audio/acoustic chain: challenges and solutions. Audio Engineering Society Conference: 23rd International Conference: Signal Processing In Audio Recording And Reproduction. Wattenberg, M., & Viegas, F. (2008). Tag clouds and the case for vernacular visualization. Interactions (New York, N.Y.), 15(4), 49–52. doi:10.1145/1374489.1374501 Wessel, D., & Wright, M. (2002). Problems and prospects for intimate musical control of computers. Computer Music Journal, 26(3), 11–22. doi:10.1162/014892602320582945 Zawacki, L. F., & Johann, M. O. (2014). Analogue audio recording using remote servers. In D. Keller, V. Lazzartini, & M. S. Pimenta (Eds.), Ubiquitous Music (pp. 83–107). Springer. doi:10.1007/978-3-319-11152-0_5 eNdNoTeS 1. 2. 3. 18 Throughout this text we avoid the term “user”, featured in many technology-oriented essays, because of its multiple negative meanings. By replacing the term “doer” with “user”, a passive attitude toward technology is enforced. One implication is that infrastructure design is the exclusive right of an elite that gives technological assets to “consumers”. Users and consumers are terms championed by two organisations: information-technology corporations and drug cartels. The acoustic-instrumental framework is characterised by a hierarchical and centralised temporal organisation, by a strict split between audience and creators, and by notions tied to instrumental playing such as the orchestra, the note and the metric system. An illustrative example of this corporate lingo is the defence of Non-Fungible Tokens laid out by Ethereum: “Naysayers often bring up the fact that NFTs ‘are dumb’, usually alongside a picture of them screenshotting an NFT artwork. ‘Look, now I have that image for free!’ they say smugly. Well, yes. But does googling an image of Picasso’s Guernica make you the proud new owner of a multi-million dollar piece of art history? Ultimately owning the real thing is as valuable as the market makes it. The more a piece of content is screen-grabbed, shared, and generally used, the more value it gains. Owning the verifiably real thing will always have more value than not” (Ethereum n.d.) International Journal of Software Innovation Volume 12 • Issue 1 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. We address this critique especially to the past stages of the evolution of blockchain and NFTs, whereby a series of complex and energy-expensive procedures (proof-of-work mechanisms) was fundamental in order to validate transactions. Ethereum has recently alleviated this outrageous situation of worrying unsustainability through a move dubbed as “The Merge”, that replaced proof-of-work with proof-of-stake (https://ethereum.org/en/roadmap/merge/) Acoustic-instrumental music theory describes rhythm as a succession of notational figures, often based on a metre-based organisation with the concept of beat at its core. soumix.com.br www.ujam-studio.com https://www.izotope.com/en/products/mix/neutron.html According to Viégas and Wattenberg (2008), the concept of tag cloud was introduced by the social psychologist Stanley Milgram and his team in a study targeting the collective perception of Paris landmarks (Milgram and Jodelet 1976). In Milgram’s data visualisations, the font size was correlated to how frequently each place was mentioned by the subjects. Tag clouds have been used for various purposes, but their most common application is to provide a visual display of relationships among words. For a yet earlier formulation of the concept, see Minsky (1980). https://www.youtube.com/watch?v=W0NfraK6tSk&t=174s These sounds do not fit well within the standard categories. They tend to be described as “eccentric” since their variability does not fit within the acoustic-instrumental classifications of timbre. Dack discusses Shaefferean tipologies highlighting the limitations of this proposal: “The large note is a continuous, coherent, evolving sound of medium duration. The fragment has (as one would expect) a short duration but not as short as an impulse. Its mass is perceived as disordered and incoherent. Lastly, the cell is of medium duration and consists of a number of short, discrete component sounds” (Dack 2001: 5). https://www.youtube.com/watch?v=LiNb_T8oluA&t=219s Samples can be accessed directly from the Freesound Library here: https://freesound.org/people/tender_ buttons/ A popular shorthand for the number of items that can be handled in short-term memory is 7 (plus or minus 2). When dealing with time-based data, such as sound, we also need to consider that each item requires a larger amount of cognitive resources than, let’s say, retrieving and processing one character of a typed message. Consequently, the cognitive limitations imposed by this task may be much tighter than the standard examples considered by the psychophysics literature. See discussion on early domain restriction is (Lima et al. 2012). Marcello Messina is a Sicilian composer and academic working as Chief Researcher at Southern Federal University (SFEDU). He holds a Ph.D. in composition from the University of Leeds (UK), and is also holding honorary positions as Visiting Professor at the Federal Universities of Paraíba and Acre. He has been the recipient of the Endeavour Research Fellowship at Macquarie University, Sydney, Australia, and of the PNPD/Capes post-doctoral bursary at the Universidade Federal do Acre, Brazil. Ariane Stolfi is architect, designer, musician and lecturer on Arts and Communication on the Federal University of South of Bahia. She has been researching interactive interfaces on web technologies since 2005 and develops installations, performances and digital music instruments. Develops Playsound.space platform and coordinates Reverbera! project of free improvisation music. 19