WorldKit: rapid and easy creation of ad-hoc interactive applications on everyday surfaces

Instant access to computing, when and where we need it, has long been one of the aims of research areas such as ubiquitous computing. In this paper, we describe the WorldKit system, which makes use of a paired depth camera and projector to make ordinary surfaces instantly interactive. Using this system, touch-based interactivity can, without prior calibration, be placed on nearly any unmodified surface literally with a wave of the hand, as can other new forms of sensed interaction. From a user perspective, such interfaces are easy enough to instantiate that they could, if desired, be recreated or modified "each time we sat down" by "painting" them next to us. From the programmer's perspective, our system encapsulates these capabilities in a simple set of abstractions that make the creation of interfaces quick and easy. Further, it is extensible to new, custom interactors in a way that closely mimics conventional 2D graphical user interfaces, hiding much of the complexity of working in this new domain. We detail the hardware and software implementation of our system, and several example applications built using the library.