Ambiguous Devices: Improvisation, agency, touch and feedthrough in distributed music performance

This article documents the processes behind our distributed musical instrument, Ambiguous Devices. The project is motivated by our mutual desire to explore disruptive forms of networked musical interactions in an attempt to challenge and extend our practices as improvisers and instrument makers. We begin by describing the early design stage of our performance ecosystem, followed by a technical description of how the system functions with examples from our public performances and installations. We then situate our work within a genealogy of human–machine improvisation, while highlighting specific values that continue to motivate our artistic approach. These practical accounts inform our discussion of tactility, proximity, effort, friction and other attributes that have shaped our strategies for designing musical interactions. The positive role of ambiguity is elaborated in relation to distributed agency. Finally, we employ the concept of ‘feedthrough’ as a way of understanding the co-constitutive behaviour of communication networks, assemblages and performers.

Lessons From Joint Improvisation Workshops for Musicians and Robotics Engineers

We report on a series of workshops with musicians and robotics engineers aimed to study how human and machine improvisation can be explored through interdisciplinary design research. In the first workshop, we posed two leading questions to participants. First, what can AI and robotics learn by how improvisers think about time, space, actions, and decisions? Second, how can improvisation and musical instruments be enhanced by AI and robotics? The workshop included sessions led by the musicians, which provided an overview of the theory and practice of musical improvisation. In other sessions, AI and robotics researchers introduced AI principles to the musicians. Two smaller follow-up workshops comprised of only engineering and information science students provided an opportunity to elaborate on the principles covered in the first workshop. The workshops revealed parallels and discrepancies in the conceptualization of improvisation between musicians and engineers. These thematic differences could inform considerations for future designers of improvising robots.

Aesthetics, Interaction and Machine Improvisation

Departing from the artistic research project Goodbye Intuition (GI) hosted by the Norwegian Academy of Music in Oslo, this article discusses the aesthetics of improvising with machines. Playing with a system such as the one described in this article, with limited intelligence and no real cognitive skills, will obviously reveal the weaknesses of the system, but it will also convey part of the preconditions and aesthetic frameworks that the human improviser brings to the table. If we want the autonomous system to have the same kind of freedom we commonly value in human players’ improvisational practice, are we prepared to accept that it may develop in a direction that departs from our original aesthetical ambitions? The analyses is based on some of the documented interplay between the musicians in a group in workshops and laboratories. The question of what constitutes an ethical relationship in this kind of improvisation is briefly discussed. The aspect of embodiment emerges as a central obstacle in the development of musical improvisation with machines.

Probabilistic Factor Oracles for Multidimensional Machine Improvisation

This article presents two methods to generate automatic improvisation using training over multidimensional sequences. We consider musical features such as melody, harmony, timbre, etc., as dimensions. We first present a system combining interpolated probabilistic models with a factor oracle. The probabilistic models are trained on a corpus of musical work to learn the correlation between dimensions, and they are used to guide the navigation in the factor oracle to ensure a logical improvisation. Improvisations are therefore created in a way in which the intuition of a context is enriched with multidimensional knowledge. We then introduce a system creating multidimensional improvisations based on communication between dimensions via probabilistic message passing. The communication infers some anticipatory behavior on each dimension influenced by the others, creating a consistent multidimensional improvisation. Both systems were evaluated by professional improvisers during listening sessions. Overall, the systems received good feedback and showed encouraging results—first, on how multidimensional knowledge can improve navigation in the factor oracle and, second, on how communication through message passing can emulate the interactivity between dimensions or musicians.

Design of an Automatic Music Transcription System for the Traditional Repertoire of the Marovany Zither from Madagascar

This chapter presents an original approach for the development of an automatic music transcription system of a Malagasy traditional plucked string instrument, called marovany zither. Our approach is based on a technology of multichannel capturing sensory system, which allows breaking down a complex polyphonic audio signal into a sum of monophonic sensor signals. A very high precision in transcription is obtained, i.e. & gt; 95% on the average note-based F-measure metric. A second part of this chapter consists in using these transcripts in the human-machine improvisation system ImproteK. Details of an exploratory working session with a local Malagasy musician are reported and discussed.

Listening as a Creative Act

Some important theories of music cognition, such as Lerdahl and Jackendoff’s (1983)A Generative Theory of Tonal Music, posit an archetypal listener with an ideal interpretation of musical structure, and many studies of the perception of structure focus on what different listeners have in common. However, previous experiments have revealed that listeners perceive musical structure differently, depending upon their music background and their familiarity with the piece. It is not known what other factors contribute to differences among listeners’ formal analyses, but understanding these factors may be essential to advancing our understanding of music perception.We present a case study of two listeners, with the goal of identifying the differences between their analyses, and explaining why these differences arose. These two listeners analyzed the structure of three performances, a set of improvised duets. The duets were performed by one of the listeners and Mimi (Multimodal Interaction for Musical Improvisation), a software system for human-machine improvisation. The ambiguous structure of the human-machine improvisations as well as the distinct perspectives of the listeners ensured a rich set of differences for the basis of our study.We compare the structural analyses and argue that most of the disagreements between them are attributable to the fact that the listeners paid attention to different musical features. Following the chain of causation backwards, we identify three more ultimate sources of disagreement: differences in the commitments made at the outset of a piece regarding what constitutes a fundamental structural unit, differences in the information each listener had about the performances, and differences in the analytical expectations of the listeners.