Mapping 2D input to 3D immersive spatial augmented reality

This poster presents Viewpoint Cursor, a technique for mapping 2D user input from devices such as mobile phones, trackballs, or computer mice, to 3D multi-projector spatial augmented reality systems. While the ubiquity of input devices such as these make them obvious choices for spatial augmented reality, their 2D nature makes them difficult to use. Existing VR techniques rely on a display in front of the user's eyes on which to place virtual information. Immersive spatial augmented reality systems allow users to experience and interact with projected virtual information from any angle, using arbitrary placement of projectors. Viewpoint Cursor addresses these issues by mapping 2D input to a plane in front of the user's view. Ray casting is then used to find the 3D location for the cursor in the scene, which is then projected using the projection system. The user's position is tracked, with the input remapped accordingly, resulting in 2D input that matches what the user expects, regardless of their location.