A real-time human movement transfer system for an omnidirectional display

Abstract

This paper proposes a system in which a human image with movement is captured by a multiviewpoint camera and is converted in real time to three-dimensional data, which are sent by network transfer so that the image can be displayed at a remote site in three dimensions, using an omnidirectional display device. In the proposed system, the target human image is reconstructed in voxelwise, using parallel volume intersection for multiple images. The reconstructed voxels represent only the presence or absence of the object. Then coloring is required, based on texture information that is obtained by projecting the voxel on each image plane. The judgment as to whether the voxel is visible is important in the coloring process. In this study we attempt to achieve real-time processing from imaging to display. A parallel coloring procedure is developed by using a human image buffer in each camera, so that the visibility of the voxels is judged in real time. In order to investigate the effectiveness of the system, an experimental system for multiviewpoint image processing was constructed using five cameras and six PCs. A human image with movement was captured and a performance of 2.63 FPS was achieved for reconstruction with a spatial resolution of 96×96×96 (a voxel size of 10 mm). The reconstructed data were directly transferred to the omnidirectional display device without being stored in memory, and were displayed, demonstrating the effectiveness of the proposed method. © 2007 Wiley Periodicals, Inc. Electron Comm Jpn Pt 2, 90(12): 77–86, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjb.20416