Plausible Auditory Augmentation of Physical Interaction

Abstract

Interactions with physical objects usually evoke sounds, i.e., auditory feedback that depends on the interacting objects (e.g., table, hand, or pencil) and interaction type (e.g., tapping or scratching). The continuous real-time adaptation of sound during interaction enables the manipulation/refinement of perceived characteristics (size, material) of physical objects. Furthermore, when controlled by unrelated external data, the resulting ambient sonifications can keep users aware of changing data. This article introduces the concept of plausibility to the topic of auditory augmentations of physical interactions, aiming at providing an experimentation platform for investigating surface-based physical interactions, understanding relevant acoustic cues, redefining these via auditory augmentation / blended sonification and particularly to empirically measure the plausibility limits of such auditory augmentations. Besides conceptual contributions along the trade-off between plausibility and usability, a practical experimentation system is introduced, together with a very first qualitative pilot study.