A Comprehensive Study on Geometric, Appearance, and Deep Feature based Methods for Automatic Facial Expression Recognition

Abstract

Facial Expression (FE) is one kind of communication that, despite its non-verbal nature, predates verbal communication in terms of both its genesis and its conception. Automatic Facial Expression Recognition (AFER) is a predominant facet in analyzing facial images and thus has been an in-demand research problem for decades in the emerging field of Computer Vision (CV) & Artificial Intelligence (AI). Recent works on AFER systems focused on the following issues: insufficient training data which causes overfitting; robustness to identity bias, illumination & head pose variation, partial occlusion; generalization power; transformation from controlled to uncontrolled environments; cross dataset experiments. A comprehensive study on existing methods for the design and development of AFER systems is presented in the proposed study. The benchmark datasets and its characteristics are summarized. The advantages and limitations of the existing methods to extract the highly discriminative and abstract distributions are discussed. The evaluation methods to assess the performance of AFER systems, along with comparative analysis of various methods implemented on benchmark datasets are summarized. Furthermore, unresolved challenging issues in the field of AFER are presented in detail, which serves as an open-ended research area concerning the AFER problem.