Beyond: collapsible input device for direct 3D manipulation beyond the screen

Abstract

What would it be like to reach into a screen and manipulate or design virtual objects as in real world? We present Beyond, a collapsible input device for direct 3D manipulation. When pressed against a screen, Beyond collapses in the physical world and extends into the digital space of the screen, such that users can perceive that they are inserting the tool into the virtual space. Beyond allows users to directly interact with 3D media, avoiding separation between the users' input and the displayed 3D graphics without requiring special glasses or wearables, thereby enabling users to select, draw, and sculpt in 3D virtual space unfettered. We describe detailed interaction techniques, implementation and application scenarios focused on 3D geometric design and prototyping.