Recognizing the sentiments of web images using hand-designed features

Abstract

Recently, understanding sentiment expressed in social images and multimedia has attracted increasing attention by researchers. For sentiment analysis of social image, we should identify the visual features with high relations to human sentiments and then conduct analysis based on such visual features. Here, two visual vocabularies are built from color compositions and SIFT (scale-invariant feature transform) descriptors. Thereafter, the pLSA (probabilistic latent semantic analysis)-learning is employed to predict the human sentiment hidden in social images from visual words. The proposed system was evaluated to the images collected from Photo.net and Google and 15 Kobayashi's sentiments were considered to label the images. The results were compared with man-labeled ground truth and then the proposed method shows the performance with an F1-measure results of above 70%.