Sonification as a Composition Technique and Means of Artistic Expression

"Despite the ever-expanding variety of technological means accessible to the composers, performers, orchestrators, music producers, and music pedagogues of our time, the essence of music composing has mostly remained the same for centuries. The key question is: how does one form an artistically-innovative core idea of a musical piece, and to find the most ideal instruments for it?

This paper discusses composers’ explorations on establishing their individual voices, as well as finding and selecting the tools and techniques that would best serve their artistic goals in today´s complex and pluralistic network of aesthetics. To facilitate this, various software solutions have come to the market, enabling a wider palette of sounds to be used as musical “raw material”. Instead of pre-composed music based on the rhythms and pitches organised by the composer, we will focus on how to translate other kinds of data into “notes”, or more generally “sound events”, and how to use it in an artistically meaningful manner – resulting into complete musical compositions on their own right. To do this, we will present compositional case studies based on sonification, realised by various methods and technologies. These include, for example, application of the ORCID software in artistic practice. We will present Maria Kallionpaa´s environmentally themed work “El Canto del Mar Infinito” (2020), as well as her composition “The Reef” (2023), the musical material of which is based on the computer-based analysis of the sounds recorded on a coral reef. Moreover, we will discuss Olga Neuwirth´s work “Kloing!”, which drew part of its material from the seismic data gathered in the Sumatran area during the seaquake that caused the tsunami disaster on the 26th of December 2004.

Sonification techniques do not produce musically utilisable aesthetics per se. This raises the question for the creative domains in which composers engage when involving sonification techniques in their process. These domains include :

• The choice of the data or subject being sonified.

• The preprocessing of this data (e.g. the quantization of a series of floating point values to integer pitch values and metronomic timings), a specifically musical design of the sonification method, i.e. the transformation or mapping of the data to sound structures.

• A purposeful work with the parameters of the sonification.

A further approach has been explored by the TouchNoise system (2014-2017), developed by Axel Berndt, Nadia Al-Kassab and Raimund Dachselt. It is a sonification of a particle simulation. The compositional domain here is spanned primarily by a palette of interaction techniques with the particles field, including direct manipulations of the particle distribution as well as flow field and flocking algorithms. We will discuss the aesthetics that this approach evokes.

Authors: Dr. Maria Kallionpaa, Prof. Dr.-Ing. Axel Berndt, Mag. Rer. Nat., Dipl.-Psy. Hans-Peter Gasselseder"