Detection of Emotions in Artworks Using a Convolutional Neural Network Trained on Non-Artistic Images: A Methodology to Reduce the Cross-Depiction Problem
César González-Martín, Miguel Carrasco, Thomas Gustavo Wachter Wielandt
{"title":"Detection of Emotions in Artworks Using a Convolutional Neural Network Trained on Non-Artistic Images: A Methodology to Reduce the Cross-Depiction Problem","authors":"César González-Martín, Miguel Carrasco, Thomas Gustavo Wachter Wielandt","doi":"10.1177/02762374231163481","DOIUrl":null,"url":null,"abstract":"This research is framed within the study of automatic recognition of emotions in artworks, proposing a methodology to improve performance in detecting emotions when a network is trained with an image type different from the entry type, which is known as the cross-depiction problem. To achieve this, we used the QuickShift algorithm, which simplifies images’ resources, and applied it to the Open Affective Standardized Image (OASIS) dataset as well as the WikiArt Emotion dataset. Both datasets are also unified under a binary emotional system. Subsequently, a model was trained based on a convolutional neural network using OASIS as a learning base, in order to then be applied on the WikiArt Emotion dataset. The results show an improvement in the general prediction performance when applying QuickShift (73% overall). However, we can observe that artistic style influences the results, with minimalist art being incompatible with the methodology proposed.","PeriodicalId":45870,"journal":{"name":"Empirical Studies of the Arts","volume":null,"pages":null},"PeriodicalIF":1.5000,"publicationDate":"2023-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Empirical Studies of the Arts","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1177/02762374231163481","RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"0","JCRName":"HUMANITIES, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0
Abstract
This research is framed within the study of automatic recognition of emotions in artworks, proposing a methodology to improve performance in detecting emotions when a network is trained with an image type different from the entry type, which is known as the cross-depiction problem. To achieve this, we used the QuickShift algorithm, which simplifies images’ resources, and applied it to the Open Affective Standardized Image (OASIS) dataset as well as the WikiArt Emotion dataset. Both datasets are also unified under a binary emotional system. Subsequently, a model was trained based on a convolutional neural network using OASIS as a learning base, in order to then be applied on the WikiArt Emotion dataset. The results show an improvement in the general prediction performance when applying QuickShift (73% overall). However, we can observe that artistic style influences the results, with minimalist art being incompatible with the methodology proposed.
期刊介绍:
Empirical Studies of the Arts (ART) aims to be an interdisciplinary forum for theoretical and empirical studies of aesthetics, creativity, and all of the arts. It spans anthropological, psychological, neuroscientific, semiotic, and sociological studies of the creation, perception, and appreciation of literary, musical, visual and other art forms. Whether you are an active researcher or an interested bystander, Empirical Studies of the Arts keeps you up to date on the latest trends in scientific studies of the arts.