Real-time human movement mapping to a virtual environment

Abstract

The development of Virtual Reality (VR) into consumer products has given new perspectives to many areas. This paper presents a method to enhance the VR experience by mapping real time body movements of a person to a virtual world such that the user can do basic interactions (move, lift, throw) with objects found in the virtual environment. This technique combines a Kinect controlled full body avatar with Oculus Rift head-mounted-display through the Unity game engine. Despite few failures in Kinect skeletal tracker when in sideways poses, movements were mapped with zero latency allowing the user to interact in real time.