Vision-based human motion tracking using head-mounted cameras and fixed cameras

Abstract

In this paper, we propose a method to track human motions by using multiple-viewpoint images taken by both mobile cameras and fixed-viewpoint cameras. In a vision-based system, the variety of viewpoints is important for effective detection/tracking of target motions. As image sequences observed from various viewpoints contain rich information regarding entire scenes, these images are also useful for video-based analysis of human behaviors/interactions. We employ multiple numbers of mobile cameras mounted on human heads and fixed-viewpoint cameras to observe a human interaction scene. In our method, 3D human positions are tracked by using both mobile and fixed-viewpoint camera images. Transitions of human and background positions in images are estimated by using the tracked 3D positions, and the positions and poses of the mobile camera are determined by comparing the observed images with the estimation. Consequently, our method can provide position information for all humans and head motions for the humans wearing head-mounted cameras. Experimental results show the effectiveness of the proposed method. © 2007 Wiley Periodicals, Inc. Electron Comm Jpn Pt 2, 90(2): 40–53, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjb.20331