Personality prediction via multi-task transformer architecture combined with image aesthetics

IF 0.7 3区文学 0 HUMANITIES, MULTIDISCIPLINARY

Digital Scholarship in the Humanities Pub Date : 2024-06-22 DOI:10.1093/llc/fqae034

Shahryar Salmani Bajestani, Mohammad Mahdi Khalilzadeh, Mahdi Azarnoosh, Hamid Reza Kobravi

{"title":"Personality prediction via multi-task transformer architecture combined with image aesthetics","authors":"Shahryar Salmani Bajestani, Mohammad Mahdi Khalilzadeh, Mahdi Azarnoosh, Hamid Reza Kobravi","doi":"10.1093/llc/fqae034","DOIUrl":null,"url":null,"abstract":"Social media has found its path into the daily lives of people. There are several ways that users communicate in which liking and sharing images stands out. Each image shared by a user can be analyzed from aesthetic and personality traits views. In recent studies, it has been proved that personality traits impact personalized image aesthetics assessment. In this article, the same pattern was studied from a different perspective. So, we evaluated the impact of image aesthetics on personality traits to check if there is any relation between them in this form. Hence, in a two-stage architecture, we have leveraged image aesthetics to predict the personality traits of users. The first stage includes a multi-task deep learning paradigm that consists of an encoder/decoder in which the core of the network is a Swin Transformer. The second stage combines image aesthetics and personality traits with an attention mechanism for personality trait prediction. The results showed that the proposed method had achieved an average Spearman Rank Order Correlation Coefficient (SROCC) of 0.776 in image aesthetic on the Flickr-AES database and an average SROCC of 0.6730 on the PsychoFlickr database, which outperformed related SOTA (State of the Art) studies. The average accuracy performance of the first stage was boosted by 7.02 per cent in the second stage, considering the influence of image aesthetics on personality trait prediction.","PeriodicalId":45315,"journal":{"name":"Digital Scholarship in the Humanities","volume":"29 1","pages":""},"PeriodicalIF":0.7000,"publicationDate":"2024-06-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Digital Scholarship in the Humanities","FirstCategoryId":"98","ListUrlMain":"https://doi.org/10.1093/llc/fqae034","RegionNum":3,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"0","JCRName":"HUMANITIES, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

Social media has found its path into the daily lives of people. There are several ways that users communicate in which liking and sharing images stands out. Each image shared by a user can be analyzed from aesthetic and personality traits views. In recent studies, it has been proved that personality traits impact personalized image aesthetics assessment. In this article, the same pattern was studied from a different perspective. So, we evaluated the impact of image aesthetics on personality traits to check if there is any relation between them in this form. Hence, in a two-stage architecture, we have leveraged image aesthetics to predict the personality traits of users. The first stage includes a multi-task deep learning paradigm that consists of an encoder/decoder in which the core of the network is a Swin Transformer. The second stage combines image aesthetics and personality traits with an attention mechanism for personality trait prediction. The results showed that the proposed method had achieved an average Spearman Rank Order Correlation Coefficient (SROCC) of 0.776 in image aesthetic on the Flickr-AES database and an average SROCC of 0.6730 on the PsychoFlickr database, which outperformed related SOTA (State of the Art) studies. The average accuracy performance of the first stage was boosted by 7.02 per cent in the second stage, considering the influence of image aesthetics on personality trait prediction.

查看原文本刊更多论文

通过多任务转换器架构结合图像美学进行人格预测

社交媒体已进入人们的日常生活。用户有多种交流方式，其中喜欢和分享图片的方式最为突出。用户分享的每张图片都可以从审美和个性特征的角度进行分析。最近的研究证明，个性特征会影响个性化图片美学评估。在本文中，我们从不同的角度研究了相同的模式。因此，我们评估了形象美学对人格特质的影响，以检查在这种形式下它们之间是否存在任何关系。因此，在一个两阶段的架构中，我们利用图像美学来预测用户的个性特征。第一阶段包括多任务深度学习范式，由编码器/解码器组成，其中网络的核心是 Swin Transformer。第二阶段将图像美学和个性特征结合起来，利用注意力机制进行个性特征预测。结果表明，所提出的方法在 Flickr-AES 数据库的图像美学方面取得了平均 0.776 的斯皮尔曼秩相关系数（SROCC），在 PsychoFlickr 数据库的图像美学方面取得了平均 0.6730 的斯皮尔曼秩相关系数（SROCC），优于相关的 SOTA（艺术水平）研究。考虑到图像美学对人格特质预测的影响，第一阶段的平均准确率在第二阶段提高了 7.02%。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Digital Scholarship in the Humanities Multiple-

CiteScore

1.80

自引率

25.00%

发文量

期刊介绍： DSH or Digital Scholarship in the Humanities is an international, peer reviewed journal which publishes original contributions on all aspects of digital scholarship in the Humanities including, but not limited to, the field of what is currently called the Digital Humanities. Long and short papers report on theoretical, methodological, experimental, and applied research and include results of research projects, descriptions and evaluations of tools, techniques, and methodologies, and reports on work in progress. DSH also publishes reviews of books and resources. Digital Scholarship in the Humanities was previously known as Literary and Linguistic Computing.