Mobile devices for interaction in immersive virtual environments

Gamepads and 3D controllers are the main controllers used in most Virtual Environments. Despite being simple to use, these input devices have a number of limitations as fixed layout and difficulty to remember the mapping between buttons and functions. Mobile devices present interesting characteristics that might be valuable in immersive environments: more flexible interfaces, touchscreen combined with onboard sensors that allow new interaction and easy acceptance since these devices are used daily by most users. The work described in this article proposes a solution that uses mobile devices to interact with Immersive Virtual Environments for selection and navigation tasks. The proposed solution uses the mobile device camera to track the Head-Mounted-Display position and present a virtual representation of the mobile device screen; it was tested using an Immersive Virtual Museum as use case. Based on this prototype, a study was performed to compare controller based and mobile based interaction for navigation and selection showing that using mobile devices is viable in this context and offers interesting interaction opportunities.