Facial Expression Emotion Recognition Based on Transfer Learning and Generative Model

Facial expression emotion recognition has been a popular research topic, which played an important role in assisting the natural human-machine conversation. The conventional method for emotion estimation from facial expressions is to learn a CNN-based image classification model from scratch, However, learning such model requires a large number of labeled facial expression images, which is still a limited resource until now. To solve this problem, we propose a data augmentation method based on StyleGAN2 to generate artificial expression images with respect to seven emotions and use them as the additional training data. We further train an expression emotion recognition model based on a VGG16 network through transfer learning. In this research, we proposed a method using transfer learning and augmented images of facial expressions using trained VGG16 and StyleGAN2 and conducted experiments to achieve higher recognition accuracy for racial expression emotion recognition. Our experiment based on the CFEE dataset suggested that an emotion recognition accuracy of 75.10% could be obtained through transfer learning and the accuracy could further improved to 82.04% with the augmented expression images.