Audiovisual granular synthesis: micro relationships between sound and image

Abstract

The real-time audio-visual instrument presented in this paper enables a unified environment for composition and performance through audio-visual granular synthesis. The instrument empowers the user to reconstruct grains into new experiences and make possible real-time improvisation of the material at the granular level. The user can form audio-visual relationships through interacting with and observing perceptual similarities between sound and image at the micro and macro level. These relationships provide the user with a unified parameter for audio-visual manipulation. Throughout the paper I discuss and illustrate the design of the instrument, its performance potential and its innovative application in the field of interactive entertainment, particularly in the performance work of DJs and VJs.