Dynamic gaze analysis: An application enviroment for face-to-face communication

Abstract

Gaze analysis in dynamic environments has remained an unresolved problem due to the complexities that pertain to the detection and tracking of objects in the visual environment. This study provides a solution to the problem for face-to-face communication, in which the visual objects in the environment are faces. The application that has been developed for this purpose is able to detect and track faces in video steram, and it maps gaze locations to the images, thus allowing the user to detect gaze behavior, such as gaze aversion. The application is also capable of segmentation and diarization of speech syncronously with the vide o stream and eye movement overlay. It allows the user to annotate speech by speech act labels. The pilot studies reveal that the application provides acceptable accuracy values in the analysis, as well as significantly reducing the time for the analyses.