3D Face Recognition using Photometric Stereo and Deep Learning

Abstract

Illumination variance is one of the largest real-world problems when deploying face recognition systems. Over the last few years much work has gone into the development of novel 3D face recognition methods to overcome this issue. Photometric stereo is a well-established 3D reconstruction technique capable of recovering the normals and albedo of a surface. Although it provides a way to obtain 3D data, the amount of training data available captured using photometric stereo often does not provide sufficient modelling capacity for training state-of-the-art feature extractors, such as deep convolutional neural networks, from scratch. In this work we present a novel approach to utilising the lighting apparatus commonly used for photometric stereo to synthesise data that can act as a biometric. Combining this with deep learning techniques not only did we achieve near state-of-the-art results, but it gave insight into the possibility of using photometric stereo without the need of reconstruction. This could not only simplify the face recognition process but avoid unnecessary error that may arise from reconstruction. Additionally, we utilise the active lighting from photometric stereo to evaluate the effect of illumination on face recognition. We compare our method to the state-of-the-art 3D methods and discuss potential use cases for our system.