Regularized Least-Squares Coding with Unlabeled Dictionary for Image-Set Based Face Recognition

Image set based face recognition provides more opportunities compared to single mug-shot face recognition. However, modelling the variations in an image set is a challenging task. We propose a computationally efficient and accurate image set modelling technique. The idea is to reconstruct each image set sample with an unlabeled dictionary using the computationally efficient regularized least squares. The reconstruction coefficients form a latent representation of an image set and efficiently model its underlying structure. We propose max and sum pooling to aggregate the latent representations into a single compact feature vector representation per set. We then perform Linear Discriminant Analysis on the pooled reconstruction coefficients to increase the discrimination and reduce the dimensionality of the proposed features. The proposed algorithm is extensively evaluated for the task of image set based face recognition on the Honda/UCSD, CMU Mobo and YouTube celebrities datasets. Experimental results show that the proposed algorithm outperforms current state-of-the-art image set classification algorithms in terms of both accuracy and execution time.