Aesthetic and auditory enhancements for multi-stream information sonification

Abstract

Sonification is an emerging modality of information representation, the auditory equivalent of visualization employing non-speech sound to display attributes of form, pattern, recurrence and trends in abstract data. Like data-art or visual and auditory art-forms driven by data content directly mapped to their rendering, sonification shares the goal of aesthetic representation (auditory graphing) in a way to better and more accessibly convey the message to broader consumer audiences. Often, the simple re-contextualization of dense abstract data in an auditory graph (or sonification) is sufficient to highlight long-term trends, to hear regularities (patterns) and anomalies in periodicity of time-series data and to assimilate very subtle and fine transformations. Sonification is also optimal for certain working or ambient situations that are visually rich or visually saturated, when we seek to command topical and peripheral attention with relevant cues. Auditory display is also an alternative to visualization for people with visual impairments. Exploring the premise that sonification should be both aesthetic and informative, i.e. listenable, attractive and engaging, this paper summarises the findings of 3 experiments conducted to determine ways to better represent and access dense information mapped on to more than one concurrent stream of information. Specifically, we show evidence that spatialization of informative events coinciding in time can be more clearly distinguished and that timbre (or tone colour / tone quality) characteristics can serve to further reinforce spatial and stream separation. These findings combine to develop comprehensible methods for representing complex data-sets. We consider human cognition, auditory perception and audio reproduction technologies that each influence the ability to display information sonically.