A New Deep Learning Method for Multi-label Facial Expression Recognition based on Local Constraint Features

Human emotions always have been reflected by the facial expression. In recent year, the facial expression recognition has been found that it can be treated as a multi-label task and some databases (such as JAFFE, FER+, RAF-ML.) which include information of multi-label facial expression also have been utilize to address relate issue. Simultaneously, some deep learning methods also be used to solve multi-label facial expression task, such as VGG13 and Deep Bi-Manifold CNN (DBM-CNN). But there are also have many weakness such as the inaccurate recognition of multi-label expressions. To overcome this drawback, a novel Deep learning with local constraint framework, called DL- LC framework, is proposed. The proposed framework will use the MTCNN as an implement to crop the local constraints features which include the infromation of facial expression. And the ResNet18 has been applied as a backbone network to extract the feature from the global and local constraint images, which can get more details of original image after incorporating local constraints in this new framework. The effectiveness of this model has been testified through rigorous experiments in this study. Comprehensive analyses reveal that, this model is outperform the recent state-of-the-art approaches for multi-label facial expression recognition.