Facial expression recognition in the presence of occlusion using local Gabor binary patterns

Abstract

Occlusion is a big challenge for facial expression recognition (FER) in real-world situations. This study investigates three different methods of feature extraction for facial expression recognition from occluded images. The Gabor filters and the local binary pattern operator (LBP) and local Gabor binary pattern (LGBP) are used for feature extraction. Six basic facial expressions plus neutral pose are considered. The K-NN classifier with sum of absolute differences distance is used in classification phase. We consider four types of very frequently occurred occlusions in real-world situations, the eyes/mouth and upper/lower face region occlusion. The experiments carried out on JAFFE database and mismatched train-test strategy was used. Experimental results show the effectiveness and high robustness of LGBP approach under a variety of occlusion conditions and provide useful insights about the effects of occlusion on FER. Using LGBP features the average accuracy 96.25% on non-occluded images, 88.77% on eyes occluded images, 92.78% on mouth occluded images, 89.18% on lower face occluded images and 90.17% on upper face occluded images was obtained.