Single camera pointing gesture recognition for interaction in edutainment applications

Abstract

In this paper, a method for recognizing pointing gestures without markers is proposed. The video-based system uses one camera only, which observes the user in front of a large screen and identifies the 2D position pointed by him/her on this screen, his/her arm being in the fully extended position towards the screen. A GVF-snake is used in order to detect the pointing hand of the user, which is tracked in the following frames using the particle filters tracker. The center of gravity of the snake is used as a feature point and is transformed using linear transformation directly into the canvas coordinates. The method was tested on a large screen using applications designed for a wide range of different and even technically unversed users such as an image exploration for a virtual museum exhibit or intuitive interaction applications for gaming purposes. Experiments show very promising results for recognizing the pointing gestures by using a single camera.