Immersive mixed reality object interaction for collaborative context-aware mobile training and exploration

Abstract

The recent generation of mobile devices brings the necessary processing power, sensorics and other hardware to enable highly interactive and immersive mixed reality experiences. However, existing applications mainly focus on single users or the individual experience, and are often limited to basic user-device interaction. This paper discusses an immersive object interaction approach based on gestural input generated solely from the integrated camera of a mobile device. Developed within the context of the CollaTrEx framework for collaborative context-aware mobile training and exploration, it particularly allows for in-situ multiuser interaction with virtual objects. The presented approach is demonstrated in a set of prototypical game implementations for smartphones.