Gaze3D: framework for gaze analysis on 3D reconstructed scenes

An ongoing challenge with head-mounted eye-trackers is how to analyze the data from multiple individuals looking at the same scene. Our work focuses on static scenes. Previous approaches involve capturing a high resolution panorama of the scene and then mapping the fixations from all viewers onto this panorama. However such approaches are limited as they typically restrict all viewers to observe the scene from the same stationary vantage point. We present a system which incorporates user-perspective gaze data with a 3D reconstruction of the scene. The system enables the visualization of gaze data from multiple viewers on a single 3D model of the scene instead of multiple 2D panoramas. The subjects are free to move about the scene as they see fit which leads to more natural task performance. Furthermore since it is not necessary to warp the scene camera video into a flat panorama, our system preserves the relative positions of the objects in the scene during the visualization process. This gives better insight into the viewer's problem solving and search task strategies. Our system has high applicability in complex static environments such as crime scenes and marketing studies in retail stores.