Pokey: Interaction through covert structured light

Abstract

In this paper we describe a method to support interaction with a cellphone-based projector-camera system. We describe a novel approach that uses a technique known in computer vision as ldquostructured lightrdquo. It is based on projecting some special pattern of light onto a scene while imaging it with a camera. The distortions of the known pattern in the resulting image are due to the scene geometry which can be readily estimated. The main contribution of this paper is that the structure is created as consequence of the way raster-scan, laser-based micro-projectors operate, and is in fact invisible to the user. The structure of the projected light is sensed through careful synchronization within the camera-projector system and is imperceptible to the user. In this paper we describe the technique, and test it with a cellphone based application that exploits this method while providing a natural interactive environment with no additional special equipment. The system enables manual interaction with a projected user interface using only the rasterizing projector and camera that will be part of next generation cellphones.