{"title":"Personality prediction via multi-task transformer architecture combined with image aesthetics","authors":"Shahryar Salmani Bajestani, Mohammad Mahdi Khalilzadeh, Mahdi Azarnoosh, Hamid Reza Kobravi","doi":"10.1093/llc/fqae034","DOIUrl":null,"url":null,"abstract":"Social media has found its path into the daily lives of people. There are several ways that users communicate in which liking and sharing images stands out. Each image shared by a user can be analyzed from aesthetic and personality traits views. In recent studies, it has been proved that personality traits impact personalized image aesthetics assessment. In this article, the same pattern was studied from a different perspective. So, we evaluated the impact of image aesthetics on personality traits to check if there is any relation between them in this form. Hence, in a two-stage architecture, we have leveraged image aesthetics to predict the personality traits of users. The first stage includes a multi-task deep learning paradigm that consists of an encoder/decoder in which the core of the network is a Swin Transformer. The second stage combines image aesthetics and personality traits with an attention mechanism for personality trait prediction. The results showed that the proposed method had achieved an average Spearman Rank Order Correlation Coefficient (SROCC) of 0.776 in image aesthetic on the Flickr-AES database and an average SROCC of 0.6730 on the PsychoFlickr database, which outperformed related SOTA (State of the Art) studies. The average accuracy performance of the first stage was boosted by 7.02 per cent in the second stage, considering the influence of image aesthetics on personality trait prediction.","PeriodicalId":45315,"journal":{"name":"Digital Scholarship in the Humanities","volume":null,"pages":null},"PeriodicalIF":0.7000,"publicationDate":"2024-06-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Digital Scholarship in the Humanities","FirstCategoryId":"98","ListUrlMain":"https://doi.org/10.1093/llc/fqae034","RegionNum":3,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"0","JCRName":"HUMANITIES, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0
Abstract
Social media has found its path into the daily lives of people. There are several ways that users communicate in which liking and sharing images stands out. Each image shared by a user can be analyzed from aesthetic and personality traits views. In recent studies, it has been proved that personality traits impact personalized image aesthetics assessment. In this article, the same pattern was studied from a different perspective. So, we evaluated the impact of image aesthetics on personality traits to check if there is any relation between them in this form. Hence, in a two-stage architecture, we have leveraged image aesthetics to predict the personality traits of users. The first stage includes a multi-task deep learning paradigm that consists of an encoder/decoder in which the core of the network is a Swin Transformer. The second stage combines image aesthetics and personality traits with an attention mechanism for personality trait prediction. The results showed that the proposed method had achieved an average Spearman Rank Order Correlation Coefficient (SROCC) of 0.776 in image aesthetic on the Flickr-AES database and an average SROCC of 0.6730 on the PsychoFlickr database, which outperformed related SOTA (State of the Art) studies. The average accuracy performance of the first stage was boosted by 7.02 per cent in the second stage, considering the influence of image aesthetics on personality trait prediction.
期刊介绍:
DSH or Digital Scholarship in the Humanities is an international, peer reviewed journal which publishes original contributions on all aspects of digital scholarship in the Humanities including, but not limited to, the field of what is currently called the Digital Humanities. Long and short papers report on theoretical, methodological, experimental, and applied research and include results of research projects, descriptions and evaluations of tools, techniques, and methodologies, and reports on work in progress. DSH also publishes reviews of books and resources. Digital Scholarship in the Humanities was previously known as Literary and Linguistic Computing.