Analyzing Eye Movements in Interview Communication with Virtual Reality Agents

Abstract

In human-agent interactions, human emotions and gestures expressed when interacting with agents is a high-level personally trait that quantifies human attitudes, intentions, motivations, and behaviors. The virtual reality space provides a chance to interact with virtual agents in a more immersive way. In this paper, we present a computational framework to analyze human eye movements by using a virtual reality system in a job interview scene. First, we developed a remote interview system using virtual agents and implemented the system into a virtual reality headset. Second, by tracking eye movements and collecting other multimodal data, the system could better analyze human personality traits in interview communication with virtual agents, and it could better support training in people's communication skills. In experiments, we analyzed the relationship between eye gaze feature and interview performance annotated by human experts. Experimental results showed acceptable accuracy value for the single modality of eye movement in the prediction of eye contact and total performance in job interviews.