Augmented audio reality: telepresence/VR hybrid acoustic environments

Abstract

Augmented audio reality consists of hybrid presentations in which computer-generated sounds are overlayed on top of more directly acquired audio signals. We are exploring the alignability of binaural signals with artificially spatialized sources, synthesized by convolving monaural signals with left/right pairs of directional transfer functions. We use MAW (multidimensional audio windows), a NeXT-based system, as a binaural directional mixing console. Since the rearrangement of a dynamic map is used to dynamically select transfer functions, a user may specify the virtual location of a sound source, throwing the source into perceptual space, using exocentric graphical control to drive egocentric auditory display. As a concept demonstration, we muted a telephone, and then used MAW to spatialize a ringing signal at its location, putting the sonic image of the phone into the office environment. By juxtaposing and mixing 'real' and 'synthetic' audio transmissions, we are exploring the relationship between acoustic telepresence and VR presentations: telepresence manifests as the actual configuration of sources in a sound field, as perceivable by a dummyhead; VR is the perception yielded by filtering of virtual sources with respect to virtual sinks. We have conducted an experiment testing the usefulness of such a hybrid.<>