The Eyes Are the Windows to the Mind: Implications for AI-Driven Personalized Interaction

Eye-tracking has been extensively used both in psychology for understanding various aspects of human cognition, as well as in human computer interaction (HCI) for evaluation of interface design or as a form of direct input. In recent years, eye-tracking has also been investigated as a source of information for machine learning models that predict relevant user states and traits (e.g., attention, confusion, learning, perceptual abilities). These predictions can then be leveraged by AI agents to model their users and personalize the interaction accordingly. In this talk, Dr. Conati will provide an overview of the research her lab has done in this area, including detecting and modeling user cognitive skills, and affective states, with applications to user-adaptive visualizations, intelligent tutoring systems and health.