Reconciling Reduced the Sense of Ownership and Agency Toward a Virtual Keypad With Enhanced Haptic Experience Using Visually Adjusted Self-Haptics for Key-Input Interaction

Abstract

Self-Haptics is a tactile feedback technique where users utilize their own body to generate tactile sensations. This method allows the body to act purely as a prop, making it useful for virtual objects and user interfaces not directly related to the body. However, any method for maintaining the sense of ownership (SoO) and agency (SoA) of the user’s body and interaction targets that aligns with the perceptual interaction by visual cues has not been established. To address this, we present the concept of “(Not My) Self-Haptics,” which aims to resolve these inconsistencies, leveraging vision’s dominance in sensory integration. In this study, through the user’s interaction with a virtual keypad (VPad) displayed at the position of the real hand (RHand), we clarified how the user’s experience changed, including SoO and SoA toward the VPad, and how the task performance was affected by the interaction with the VPad. The results show that displaying the VHand at a misaligned position with the RHand or hiding the VHand significantly reduces SoO and SoA toward the VPad. Conversely, obtaining tactile feedback through Self-Haptics during interaction with the VPad significantly decreases task performance and increases SoO toward the VPad displayed near the VHand. Despite these unintended effects, this visual modification also significantly enhances Autotelics, Involvement, and Realism based on the Haptic Experience Inventory compared to typical mid-air interaction without tactile feedback. These findings contribute to the understanding of the interaction between visual and tactile sensations in Self-Haptics and the feasibility and practicality of “(Not My) Self-Haptics”.