Acquisition of 3D gaze information from eyeball movements using inside-out camera

Abstract

We propose a method for obtaining 3D gaze information using inside-out camera. Such information on 3D gaze points can be useful not only to clarify higher cognitive processes in humans but also to reproduce the 3D shape of an object from eyeball movement simply by gazing at the object as an extension of the visual function. Using half-mirrors, an inside-out camera can capture a person's eyeball head-on and can capture the person's visual field from a position equivalent to that of the eyeball. Here, the relationship between the gaze vector obtained from images of the eyeball and the gaze point in images capturing the visual field is expressed by a conversion equation. The 3D position of the gaze point can then be estimated by using stereo constraints in two scene cameras. In an evaluation experiment, the gaze point could be estimated with an average error of about 15 pixels, and we also showed the 3D scan path obtained by the proposed method from eyeball movement by gazing at the object.