{"title":"面向方面级多模态情感分析的知识增强异构图卷积网络","authors":"Yujie Wan, Yuzhong Chen, Jiali Lin, Jiayuan Zhong, Chen Dong","doi":"10.1016/j.csl.2023.101587","DOIUrl":null,"url":null,"abstract":"<div><p>Aspect-level multimodal sentiment analysis<span><span><span> has also become a new challenge in the field of sentiment analysis. Although there has been significant progress in the task based on image–text data, existing works do not fully deal with the implicit sentiment expression in data. In addition, they do not fully exploit the important information from external knowledge and image tags. To address these problems, we propose a knowledge-augmented heterogeneous graph convolutional network (KAHGCN). First, we propose a dynamic knowledge </span>selection algorithm to select the most relevant external knowledge, thereby enhancing KAHGCN’s ability of understanding the implicit sentiment expression in review texts. Second, we propose a </span>graph construction strategy to construct a heterogeneous graph that aggregates review text, image tags and external knowledge. Third, we propose a multilayer heterogeneous graph convolutional network to strengthen the interaction between information from external knowledge, review texts and image tags. Experimental results on two datasets demonstrate the effectiveness of the KAHGCN.</span></p></div>","PeriodicalId":50638,"journal":{"name":"Computer Speech and Language","volume":null,"pages":null},"PeriodicalIF":3.1000,"publicationDate":"2023-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A knowledge-augmented heterogeneous graph convolutional network for aspect-level multimodal sentiment analysis\",\"authors\":\"Yujie Wan, Yuzhong Chen, Jiali Lin, Jiayuan Zhong, Chen Dong\",\"doi\":\"10.1016/j.csl.2023.101587\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Aspect-level multimodal sentiment analysis<span><span><span> has also become a new challenge in the field of sentiment analysis. Although there has been significant progress in the task based on image–text data, existing works do not fully deal with the implicit sentiment expression in data. In addition, they do not fully exploit the important information from external knowledge and image tags. To address these problems, we propose a knowledge-augmented heterogeneous graph convolutional network (KAHGCN). First, we propose a dynamic knowledge </span>selection algorithm to select the most relevant external knowledge, thereby enhancing KAHGCN’s ability of understanding the implicit sentiment expression in review texts. Second, we propose a </span>graph construction strategy to construct a heterogeneous graph that aggregates review text, image tags and external knowledge. Third, we propose a multilayer heterogeneous graph convolutional network to strengthen the interaction between information from external knowledge, review texts and image tags. Experimental results on two datasets demonstrate the effectiveness of the KAHGCN.</span></p></div>\",\"PeriodicalId\":50638,\"journal\":{\"name\":\"Computer Speech and Language\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.1000,\"publicationDate\":\"2023-11-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Computer Speech and Language\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0885230823001067\",\"RegionNum\":3,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Speech and Language","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0885230823001067","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
A knowledge-augmented heterogeneous graph convolutional network for aspect-level multimodal sentiment analysis
Aspect-level multimodal sentiment analysis has also become a new challenge in the field of sentiment analysis. Although there has been significant progress in the task based on image–text data, existing works do not fully deal with the implicit sentiment expression in data. In addition, they do not fully exploit the important information from external knowledge and image tags. To address these problems, we propose a knowledge-augmented heterogeneous graph convolutional network (KAHGCN). First, we propose a dynamic knowledge selection algorithm to select the most relevant external knowledge, thereby enhancing KAHGCN’s ability of understanding the implicit sentiment expression in review texts. Second, we propose a graph construction strategy to construct a heterogeneous graph that aggregates review text, image tags and external knowledge. Third, we propose a multilayer heterogeneous graph convolutional network to strengthen the interaction between information from external knowledge, review texts and image tags. Experimental results on two datasets demonstrate the effectiveness of the KAHGCN.
期刊介绍:
Computer Speech & Language publishes reports of original research related to the recognition, understanding, production, coding and mining of speech and language.
The speech and language sciences have a long history, but it is only relatively recently that large-scale implementation of and experimentation with complex models of speech and language processing has become feasible. Such research is often carried out somewhat separately by practitioners of artificial intelligence, computer science, electronic engineering, information retrieval, linguistics, phonetics, or psychology.