A Review of Local, Holistic and Deep Learning Approaches in Facial Expressions Recognition

Abstract

In facial expression identification, algorithms with higher classification rates and lower computational costs are preferred. To achieve that, feature extraction and classification should be accurate and efficient. Feature extraction optimization involves selecting the optimal feature descriptor. Various algorithms in computer vision involve holistic, local and deep learning algorithms. Holistic algorithms analyze the whole facial image and includes algorithms like Linear Discriminant Analysis or fisherfaces, eigenfaces (PCA), Histograms of Oriented Gradients and Gray Level Co-occurrence Matrix (GLCM). Local feature descriptors involve using local facial components separately then aggregating them into a combined histogram. Local binary patterns (LBP), local directional patterns (LDP) and scale-invariant feature transform (SIFT) feature extraction algorithms have been successfully used in local feature extraction. Deep learning involves using convolutional neural networks for image analysis. The most popular models are AlexNet, VGG-Face and GoogleNet. The study evaluates computational accuracy and efficiency of the three forms of facial expression recognition namely holistic, local and deep learning algorithms. The JAFFE and CK+ datasets are used for analysis. Gabor Filters are used for preprocessing filtering of the images whilst Viola Jones OpenCV toolset is used for image visualization. The study concludes that local algorithms compete very well with deep learning algorithms in terms of accuracy but use less processing power than convolutional networks. For real time facial expression analysis with minimal processing power and need for quick response times, LBP algorithms are recommended.