Sight, sound, the chicken and the egg: Audio-visual co-dependency in music

Abstract

Amongst the modern day abundance of audio-visual media, where sounds represent everything from the swooping of virtual cameras through 3D spaces to the pressing of buttons and receiving of emails, and conversely where VJs routinely accompany live musical performance with an increasingly sophisticated language of abstract computer animation, the notion of music as a necessarily exclusively aural medium seems somewhat out of place. Psychological theories relating to the cognition of sound, in particular physical schema, accounting for the ubiquity of vertical plane pitch metaphors in most musical cultures, provide evidence of a deep-rooted spatially informed understanding of sound thus providing a common ground for both sound and vision in music. Furthermore, Western Classical composition is rife with examples of visually conceived forms from Bach’s Crab Canon (1747) to Xenakis’ architecturally inspired Metastasis (1954). However, in practice the gap between the listener’s auditory experience and the composer’s visual concept is often insurmountable. Rising to Schaeffer’s call for “Primacy to the ear!” (Schaeffer, 1967, pp. 28-30), acousmatic composers have sought to derive music exclusively from experientially verifiable criteria. However, in its pervasiveness of other musical genres, no doubt aided by technologically and commercially driven domination of the pre-recorded over the live listening experience in the latter half of the twentieth century, such an approach has lead to the neglect of visual aspects in the live performance of much art-music. This research aims to begin to redress this balance through the composition of, largely computer realised, audio-visual works whose conception arises not from a superimposition of one medium upon another, but through the very relations between the media themselves. Utilising modern computers’ ability to synchronise physical and virtual visual events with synthesised sound in real time not only affords composers an invaluable tool for enhancing listener’s perception of formal structures but also implies causal relationships between the sonic and the visual which can provide a base of intuitive understanding on which more complex formal ideas can be built.