Composite Body-Tracking: Device Abstraction Layer with Data Fusion For Virtual Reality Applications

Abstract

In a time where Virtual Reality has become increasingly popular, many companies have launched new body tracking hardware. These devices deliver various types of tracking information such as a simple full body skeleton (e.g. Microsoft Kinect v2) or a detailed hand skeleton (e.g. Leap Motion). Various device abstraction layers exist, making it easy to connect to such diverse input devices. However, when an application is connected to two or more individual input devices, these devices often provide complementary or multiple, but inconsistent tracking information. To make best use of the available data, and by that, achieving the best tracking quality, proper fusion of the information is required.Hence, we present the design and architecture of the ALVR system (Abstraction Layer for Virtual Reality). On the one hand, ALVR decouples vendor-specific interfaces from the application. On the other hand, it builds and maintains a data model representing a human skeleton. ALVR may receive data from various tracking devices. It then merges redundant tracking data and integrates unique tracking data into this data model. Thereby, application developers gain access to holistic tracking information. This integrated approach - data fusion with device abstraction - allows application developers to maximize the usage of their tracking devices while optimizing the information provided by them, e.g. for gesture recognition.