Image sentiment considering color palette recommendations based on influence scores for image advertisement

Abstract

As image-based communication proliferates, the business value of image sentiment analysis is rapidly growing, particularly in fields like advertising where consumers receive emotional cues through visual stimuli. However, most existing research on image sentiment analysis has focused more on developing sentiment classification models rather than exploring specific factors contributing to image sentiment. Therefore, this study proposes a methodology for extracting color palettes to represent image sentiments, emphasizing the significance of color palettes as highlighted in various studies. Previous approaches to color palette extraction have included heuristic methods, survey-based selection, or utilizing clustering algorithms like K-means clustering based on color frequencies in images. In this study, we calculate the influence scores of colors for classifying image sentiments and propose deriving representative sentiment-color palettes based on these scores. Initially, we train a multi-label classification model to predict the sentiment labels of images and then create datasets for distorted images where pixels corresponding to specific colors are removed. By comparing the model outputs obtained from these distorted images with the original dataset, we obtain quantitative influence scores of colors for classifying sentiment labels. Furthermore, we extract sentiment-color palettes consisting of four important colors for 30 different sentiments. Experimental results demonstrate higher evaluation scores compared to previous studies.