Detection of Emotions in Artworks Using a Convolutional Neural Network Trained on Non-Artistic Images: A Methodology to Reduce the Cross-Depiction Problem

IF 1.5 4区 心理学 0 HUMANITIES, MULTIDISCIPLINARY
César González-Martín, Miguel Carrasco, Thomas Gustavo Wachter Wielandt
{"title":"Detection of Emotions in Artworks Using a Convolutional Neural Network Trained on Non-Artistic Images: A Methodology to Reduce the Cross-Depiction Problem","authors":"César González-Martín, Miguel Carrasco, Thomas Gustavo Wachter Wielandt","doi":"10.1177/02762374231163481","DOIUrl":null,"url":null,"abstract":"This research is framed within the study of automatic recognition of emotions in artworks, proposing a methodology to improve performance in detecting emotions when a network is trained with an image type different from the entry type, which is known as the cross-depiction problem. To achieve this, we used the QuickShift algorithm, which simplifies images’ resources, and applied it to the Open Affective Standardized Image (OASIS) dataset as well as the WikiArt Emotion dataset. Both datasets are also unified under a binary emotional system. Subsequently, a model was trained based on a convolutional neural network using OASIS as a learning base, in order to then be applied on the WikiArt Emotion dataset. The results show an improvement in the general prediction performance when applying QuickShift (73% overall). However, we can observe that artistic style influences the results, with minimalist art being incompatible with the methodology proposed.","PeriodicalId":45870,"journal":{"name":"Empirical Studies of the Arts","volume":null,"pages":null},"PeriodicalIF":1.5000,"publicationDate":"2023-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Empirical Studies of the Arts","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1177/02762374231163481","RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"0","JCRName":"HUMANITIES, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

Abstract

This research is framed within the study of automatic recognition of emotions in artworks, proposing a methodology to improve performance in detecting emotions when a network is trained with an image type different from the entry type, which is known as the cross-depiction problem. To achieve this, we used the QuickShift algorithm, which simplifies images’ resources, and applied it to the Open Affective Standardized Image (OASIS) dataset as well as the WikiArt Emotion dataset. Both datasets are also unified under a binary emotional system. Subsequently, a model was trained based on a convolutional neural network using OASIS as a learning base, in order to then be applied on the WikiArt Emotion dataset. The results show an improvement in the general prediction performance when applying QuickShift (73% overall). However, we can observe that artistic style influences the results, with minimalist art being incompatible with the methodology proposed.
使用非艺术图像训练的卷积神经网络检测艺术品中的情绪:一种减少交叉描绘问题的方法
本研究是在艺术品情绪自动识别的研究框架内进行的,提出了一种方法,当网络使用不同于输入类型的图像类型进行训练时,可以提高检测情绪的性能,这被称为交叉描述问题。为了实现这一点,我们使用了QuickShift算法,该算法简化了图像资源,并将其应用于开放情感标准化图像(OASIS)数据集和WikiArt情感数据集。这两个数据集也统一在一个二元情感系统下。随后,使用OASIS作为学习基础,基于卷积神经网络训练模型,然后将其应用于WikiArt情感数据集。结果表明,当应用QuickShift时,总体预测性能有所提高(总体为73%)。然而,我们可以观察到艺术风格影响了结果,极简主义艺术与所提出的方法不相容。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
2.60
自引率
0.00%
发文量
14
期刊介绍: Empirical Studies of the Arts (ART) aims to be an interdisciplinary forum for theoretical and empirical studies of aesthetics, creativity, and all of the arts. It spans anthropological, psychological, neuroscientific, semiotic, and sociological studies of the creation, perception, and appreciation of literary, musical, visual and other art forms. Whether you are an active researcher or an interested bystander, Empirical Studies of the Arts keeps you up to date on the latest trends in scientific studies of the arts.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信