Factors affecting identification of tasks using eye gaze

Abstract

The pioneering findings of Yarbus have been replicated in recent times to decipher specific tasks from eye gaze. In this study, we focused on analyzing factors that affect task decoding using Hidden Markov Models in an experiment with different pictures and tasks. Three pictures were chosen along with four questions each for the subjects to perform several visual and cognitive tasks. Areas of interest were chosen for each picture based on the heat maps of the scanpaths from a preliminary experiment. Hidden Markov Models with discrete emissions were used for task prediction. Three variables in particular were tested - the impact of post-answer period, the impact of dwell time in Hidden Markov Models and the impact of a task being performed first in a picture. There were three sets of experiments performed in which the order of the questions was changed. Using dwell time in Hidden Markov Models showed significant improvement in success rates whereas excluding post-answer period decreased them. The analysis of the sequence of tasks of the three data sets showed that the average success rates for tasks were higher when they were seen second in the sequence than when they were seen first.