Effects of Sound Volume Change When Squeezing a Virtual Soft Object with a Bare Hand

Abstract

In order to improve the perception of the core part of a virtual soft object when a user squeezes it with his/her bare hand, we apply multisensory integration of visual and auditory stimuli to our proposed AR system. The visual stimuli are real-time stereoscopic images in which the user's bare hand is squeezing the virtual soft object in the actual scene. As the auditory stimuli, we focus on the sound volume change linked to the movements of the user's thumb and index finger. The present paper reports the effects of sound volume change in enhancing the pseudo-softness and the perception of the core part of a virtual soft object. The experimental results of our study statistically show that when a user squeezes a virtual soft object with his/her bare hand, multisensory integration of the visual and auditory stimuli effectively increases the feeling of grasping and facilitates handling of the virtual soft object. In addition, auditory stimuli that have a clearly audible sound volume change at the core part naturally enhance the perception of the core part of the virtual soft object.