Local feature hierarchy for face recognition across pose and illumination

Abstract

Even though face recognition in frontal view and normal lighting condition works very well, the performance degenerates sharply in extreme conditions. In real applications, both the lighting and pose variation will always be encountered at the same time. Accordingly we propose an end-to-end face recognition method to deal with pose and illumination simultaneously based on convolutional neural networks where the discriminative nonlinear features that are invariant to pose and illumination are extracted. Normally the global structure for images taken in different views is quite diverse. Therefore we propose to use the 1 × 1 convolutional kernel to extract the local features. Furthermore the parallel multi-stream multi-layer 1 × 1 convolution network is developed to extract multi-hierarchy features. In the experiments we obtained the average face recognition rate of 96.9% on multiPIE dataset, which improves the state-of-the-art of face recognition across poses and illumination by 7.5%. Especially for profile-wise positions, the average recognition rate of our proposed network is 97.8%, which increases the state-of-the-art recognition rate by 19%.