Real Time Light Source Estimation Using a Fish-Eye Lens with ND Filters

Abstract

3D virtual objects can be augmented in the streaming data through a webcam for various purposes using a marker and AR techniques. But the augmented results without shadows it can be shown unnaturally because the illumination of the real scene does not be considered. To minimize the unnaturalness between the background and virtual object, the illumination of the background is uniformly applied to the virtual object and real object. In this paper we propose a method to estimate light source information in real time for realistic synthesis. The proposed estimation system consists of a camera, fish-eye lens and ND (neutral density) filters. The fish-eye lens covers 180 degree of the real environment. Using these hemisphere images through the lens the direction of the lights can be estimated. As it considers the energy of pixels in the images the bright regions contain more light information than the other regions. To detect these regions image processing is additionally needed. But using iris of fish-eye lens and ND filters which reduce the incident light we can estimate the bright regions without additional image processing. The proposed method can estimate the light source information in real time and it can be used for an AR or VR environment to increase the naturalness.