3D content generation using a TOF camera and HDR light information

Abstract

We propose a natural 3D scene generation method using a TOF camera and real light information. We can generate 3D human actor by capturing a depth and color image of human actor using a TOF camera which uses the IR sensor. But the acquisition is processed in an indoor environment due to the time and space constraints of capturing environments. In practice, a composition using a 3D human actor with a real scene has some problems such as low quality depth data of the TOF camera and the absence of illumination. Especially the illumination cannot be acquired when we capture a 3D human actor. To compose a 3D human actor with a real scene, the real light information of an environment should be represented. To efficiently use the real light information we estimate a small number of light sources from the HDR radiance map using a segmentation and a sampling methods. Experimental results show that our proposed method successfully generates natural illumination effects on the reconstructed 3D actor and the virtual object by using the estimated light information.