Estimating Torso Direction Based on a Head-Mounted Display and Its Controllers for Steering During Virtual Locomotion

Abstract

Reliable estimation of torso direction is essential for virtual locomotion as it controls the locomotion direction in virtual environments. Existing approaches rely on external sensors to track torso direction or use the poses and/or movements from the head, hands, or feet to control the direction of travel. In addition, deep learning methods based on a head-mounted display (HMD) and its controllers were proposed for torso direction estimation but are unable to produce continuous estimates or that the estimation errors were not systematically evaluated. On the other hand, 3D full-body human motion generation methods based on an HMD and its controllers are capable of estimating torso direction but this capability was largely ignored for its use in virtual steering. We proposed a new method called TCNPoser that can perform torso direction estimation and generate 3D full-body human motion. Through offline evaluation and a user study, we show that the proposed method achieves the state-of-the-art (SOTA) performance for real-time torso direction estimation.