Sensing visual attention using an interactive bidirectional HMD

This paper presents a novel system for sensing of attentional behavior in Augmented Reality (AR) environments by analyzing eye movement. The system is based on light weight head mounted optical see-through glasses containing bidirectional microdisplays, which allow displaying images and eye tracking on a single chip. The sensing and interaction application has been developed in the European project ARtSENSE in order to (1) detect museum visitors attention/interest in artworks as well as in presented AR content, (2) present appropriate personalized information based on the detected attention as augmented overlays, and (3) allow museum visitors gaze-based interaction with the system or the AR content. In this paper we present a novel algorithm for pupil estimation in low resolution eye-tracking images and show first results on attention estimation by eye movement analysis and interaction with the system by gaze.