Interactive Estimation of Light Source Position and Reflectance of Real Objects for Mixed-Reality Application

The seamless integration of real and virtual objects is required for mixed-reality applications. To achieve this goal, we should represent an effect of light reflection like shading, shadowing and inter- reflection between the real and virtual objects. To represent these effects, we have to estimate reflectance of the real objects. The reflectance can be estimated with color and geometry of the objects and light condition of the scene. To calculate at an interactive frame rates, the light sources are distributed on a surface of a dome above the scene. To estimate the reflectance more accurately, we have to calculate distance from the objects to the light source. Therefore, this paper proposes a method to estimate the distance from the objects to the light source and the reflectance of the objects at an interactive frame rates. In the proposed method, two cameras and a marker with a spherical mirror are used. We can use an RGB camera and an IR camera of Microsoft Kinect sensor as the cameras. In other words, by use of the proposed method, we can estimate the distance and reflectance by using the Kinect and the marker with the spherical mirror. In the method, intersection points of reflection vectors on the spherical mirror at each camera are evaluated and the point which has the maximum evaluation value is regarded as an estimation value of the light source position. With the proposed method, we can estimate the light source position and reflectance of the real objects at an interactive frame rates by use of the Kinect and the marker with the spherical mirror, so that we are able to apply the method to various mixed-reality applications.