Shadow-aware Uncalibrated Photometric Stereo Network

Abstract

Shadow is an important clue to the relation between the scene’s structure and lighting condition. However, most photometric stereo algorithms treat shadows as outliers and thus lost this information. In this work, we propose a shadow-aware photometric stereo network that explicitly takes advantage of shadow information. We introduce a shadow estimation model to detect cast shadows and design a reconstruction loss based on the estimated shadow map. On the one hand, adding the shadow information in the reconstruction loss can effectively reduce the influence of scene shadow to normal estimation. On the other hand, the proposed shadow estimation model solves the bas-relief ambiguity problem in uncalibrated photometric stereo. Experiments show the superiority of our SAPS-Net against other uncalibrated photometric stereo algorithms. Besides, the proposed reconstruction loss makes it possible for SAPS-Net to be optimized on real-world data by fine-tuning itself in a self-supervised way, making our method more practical.