Human Face Sentiment Classification Using Synthetic Sentiment Images with Deep Convolutional Neural Networks

Abstract

Image is one of the most important ways for users to express their emotions on social networks. In this paper, we use the deep convolutional neural networks to solve the problem of image sentiment analysis from visual content. Because training a neural network requires a large number of data sets to provide good training performance, we cannot obtain such a real human emotion training set, because emotions are subjective, and multiple people need to provide annotations for the images, which requires a lot of manpower. This study proposes to incorporate synthetic face images into the training set to substantially increase the size of the training set. We use only synthetic face images, real facial images, and mixtures of synthetic and real facial images in the training set. Our experiments show that by using only 4026 real images, where each image is supplemented by the synthetic image to the same data set size (Anger: 1063 + 937 true, Disgust: 1857 + 143 true, Fear: 1802 + 198 true, Happy: 2000 true, Sad: 1252 + 748 true) total of 10,000 images, can reach 87.79%, 74.19%, 86.99%, 79.80% average testing accuracy in each testing set in human face sentiment classification.