Real-Time Localization and Matching of Corneal Reflections for Eye Gaze Estimation via a Lightweight Network

Abstract

Eye gaze estimation has lots of potentials in human-computer interaction. A popular scheme for eye gaze estimation in literature uses some infrared (IR) lights to illuminate the eyes while an IR camera captures the images, and the essential step in this scheme is to locate the glints on the cornea illuminated by the IR lights (called corneal reflections) and to match them to the corresponding IR lights. However, corneal reflections are often dim or even absent, and together with other image quality issues, the accuracy and continuity of a gaze estimation system can be severely impaired. Addressing the above problems, this paper designs a new gaze estimation hardware system, and then proposes a lightweight deep neural network for real-time localization and matching of corneal reflections, which can be simply deployed in the designed hardware system. This network, inspired by keypoint detection, can simultaneously locate corneal reflections and match them to the corresponding IR lights. It also merges the two tasks of locating the pupil center and locating the corneal reflections through an attention module. Experiments show the proposed network achieves better performances on localization and matching corneal reflections in comparison to a state-of-the-art method. And additionally, our designed system is able to provide accurate and continuous gaze estimation for real-time applications.