Projection-Based User Interface for Smart Home Environments

Abstract

Nowadays, technology has made it relatively easy to build a smart home environment. To better interact with the devices and services available in such a smart environment, the user interface plays an important role. In view of this, this paper presents a vision and projection-based user interface that allows the user to remotely control the devices through the movement of fingertips over a projected surface, thereby offering a more natural and intuitive way of user interaction. To realize the proposed bare hand interface, a real-time fingertips detection method and a fingertips tracking method are proposed, where the fingertips detection is accomplished by applying computer vision techniques to the images acquired by a camera. In addition, we propose a gesture recognition model to enable a mapping from the position of fingertips to the high level user actions and events, so that several common gestures (e.g. tap/click and drag) can be correctly detected. To demonstrate the feasibility of the proposed approach, a proof-of-concept experiment, a prototype of the projection-based user interface, is conducted and presented based on a typical smart home scenario.