Graphic-to-Sound Sonification for Visual and Auditory Communication Design

Abstract

I designed two sonification platforms designed for visual/auditory communication design studies and audiovisual art. The purpose of this study was to examine whether test participants can associate visuals and sound without any prior training and sonification approaches in this paper can be utilized as an interactive musical expression. The platform for the communication design study was developed first and the artistic audiovisual platform with the same sonification methodology followed next. In this paper, I introduce the (former) sonification platform designed for the image-to-sound association studies, their sonification methodologies, and present the study results. The object-oriented sonification method that I newly developed describes each shape sonically. The five image-sound association studies were conducted to see whether people can successfully associate sounds and fundamental shapes (i.e., a circle, a triangle, a square, lines, curves, and other custom shapes). Regardless of age and educational background, the correct answer rate was high.