Novel Application of an RGB-D Camera for Face-Direction Measurements and Object Detection: Towards Understanding Museum Visitors' Experiences

Abstract

Gaze measurement techniques can be used to understand the experiences of visitors at museums. In previous research, researchers used a wearable sensor such as an eye tracker to estimate what a visitor was looking at. Although it is possible to measure this experimentally by using a wearable sensor, it is difficult to measure the experience of all museum visitors with such an approach. Therefore, in this study, we propose a method for measuring visitors' experiences with non-wearable type sensors. In this method, we measure the face direction, which is said to be related to the gaze direction of the visitors, by using non-wearable Kinect sensors. We also have to detect objects because we need information about what people are looking at to measure the visitors' experiences. In this paper, we describe the method to estimate the gaze target of the visitor, which is referred to as the visitor's experience, based on the face-direction measurements and object detection data collected simultaneously, and an experiment is conducted to evaluate its effectiveness.