Lighting Direction Estimation of a Shaded Image by a Surface-input Regression Network

Abstract

In augmented reality (AR), the lighting direction plays an important role to the quality of the augmented scene. The corresponding lighting direction estimation is a challenging problem as it depends on an extra unknown variable -reflectance of the material. In this article, we propose to estimate the lighting direction by a neural network (NN) which is trained by a sample set. Since the empirical reflectance of a captured scene is in form of scattered points, we unify the representation of reflectance as a two dimensional polynomials. Moreover, a novel neural network model is presented to construct the mapping from reflectance to lighting direction. Contrary to the existing NNs, the proposed model accepts surface input pattern in which the drawbacks of feature vector are overcome. Experimental results of 2000 lighting estimations with unknown reflectances are presented to demonstrate the performance of the proposed algorithm.