Swiping with luminophonics

Abstract

Luminophonics is a system that aims to maximize cross-modality conversion of information, specifically from the visual to auditory modalities, with the motivation to develop a better assistive technology for the visually impaired by using image sonification techniques. The project aims to research and develop generic and highly-configurable components concerned with different image processing techniques, attention mechanisms, orchestration approaches and psychological constraints. The swiping method that is introduced in this paper combines several techniques in order to explicitly convert the colour, size and position of objects. Preliminary tests suggest that the approach is valid and deserves further investigation.