Demonstrating the Use of Rapid Touch Interaction in Virtual Reality for Prolonged Interaction in Productivity Scenarios

Abstract

Current camera-based VR headsets support free-hand mid-air inter-action or physical hand-held controllers for input, which can lead to fatigue during use, as users lack support for their arms and hands between interactions. In our demonstration, we showcase a novel approach to bring quick touch interaction to Virtual Reality, illustrating the beneficial use of rapid tapping, typing, and surface gestures for ongoing interaction in Virtual Reality, particularly in the con-text of content creation and productivity scenarios. The productivity scenarios that become possible using our approach are therefore reminiscent of apps that exist on today’s phones and tablets. To reliably make touch interaction work in VR, we use a wrist-worn prototype to complement the optical hand tracking from VR headsets with inertial sensing to detect touch events on surfaces. Our prototype band TapID integrates a pair of inertial sensors in a flexible strap, from whose signals TapID reliably detects surface touch events and identifies the finger used for touch. This event detection is then fused with the optically tracked hand poses to trigger input in VR. Our demonstration comprises a series of VR applications, including UI control in word processors, web browsers, and document editors.