Pre-trained classifiers with One Shot Similarity for context aware face verification and identification

Abstract

Most affect based systems analyse facial expressions for emotion detection, and utilize face detection and recognition methods in order to do effective affect analysis. Recent work has demonstrated the efficacy of deep architectures for face recognition by training as classifiers on voluminous datasets. Some architectures are trained as classifiers, and some directly learn an embedding via a triplet loss function. In this paper, we consider the case of one shot prediction from the feature space learnt initially via classification, i.e. we consider the situation where we have a pre-trained model, but do not have access to the training data and are required to make predictions on novel faces with just one training image per identity. We utilize the one shot similarity metric in order to compute similarity scores and compare it with the state-of-the-art results on the Youtube videos face dataset (YTF). We demonstrate the effect of temporal context on frame wise face recognition, and use a probabilistic majority voting scheme over past frames to determine current frame identity. Additionally, we found a number of labelling errors in the Youtube face dataset that were not published in the errata, and have published the same online for the benefit of the community.