Information Fusion最新文献

{"title":"Hypergraph convolutional networks with multi-ordering relations for cross-document event coreference resolution","authors":"Wenbin Zhao ,&nbsp;Yuhang Zhang ,&nbsp;Di Wu ,&nbsp;Feng Wu ,&nbsp;Neha Jain","doi":"10.1016/j.inffus.2024.102769","DOIUrl":"10.1016/j.inffus.2024.102769","url":null,"abstract":"<div><div>Recognizing the coreference relationship between different event mentions in the text (i.e., event coreference resolution) is an important task in natural language processing. It helps to understand the association between various events in the text, and plays an important role in information extraction, question answering systems, and reading comprehension. Existing research has made progress in improving the performance of event coreference resolution, but there are also some shortcomings. For example, most of the existing methods analyze the event data in the document in a serial processing mode, without considering the complex relationship between events, and it is difficult to mine the deep semantics of events. To solve these problems, this paper proposes a cross-document event co-reference resolution method (HGCN-ECR) based on hypergraph convolutional neural networks. Firstly, the BiLSTM-CRF model was used to label the semantic role of the events extracted from a number of documents. According to the labeling results, the trigger words and non-trigger words of the event were determined, and the multi-document event hypergraph was constructed around the event trigger words. Then hypergraph convolutional neural networks are used to learn higher-order semantic information in multi-document event hypergraphs, and multi-head attention mechanisms are introduced to understand the hidden features of different event relationship types by treating each event relationship as a set of separate attention mechanisms. Finally, the feed-forward neural network and the average link clustering method are used to calculate the coreference score of events and complete the coreference event clustering, and the cross-document event coreference resolution is realized. The experimental results show that the cross-document event co-reference resolution method is superior to the baseline model.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102769"},"PeriodicalIF":14.7,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142554836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multimodal dual perception fusion framework for multimodal affective analysis","authors":"Qiang Lu ,&nbsp;Xia Sun ,&nbsp;Yunfei Long ,&nbsp;Xiaodi Zhao ,&nbsp;Wang Zou ,&nbsp;Jun Feng ,&nbsp;Xuxin Wang","doi":"10.1016/j.inffus.2024.102747","DOIUrl":"10.1016/j.inffus.2024.102747","url":null,"abstract":"<div><div>The misuse of social platforms and the difficulty in regulating post contents have culminated in a surge of negative sentiments, sarcasms, and the rampant spread of fake news. In response, Multimodal sentiment analysis, sarcasm detection and fake news detection based on image and text have attracted considerable attention recently. Due to that these areas share semantic and sentiment features and confront related fusion challenges in deciphering complex human expressions across different modalities, integrating these multimodal classification tasks that share commonalities across different scenarios into a unified framework is expected to simplify research in sentiment analysis, and enhance the effectiveness of classification tasks involving both semantic and sentiment modeling. Therefore, we consider integral components of a broader spectrum of research known as multimodal affective analysis towards semantics and sentiment, and propose a novel multimodal dual perception fusion framework (MDPF). Specifically, MDPF contains three core procedures: (1) Generating bootstrapping language-image Knowledge to enrich origin modality space, and utilizing cross-modal contrastive learning for aligning text and image modalities to understand underlying semantics and interactions. (2) Designing dynamic connective mechanism to adaptively match image-text pairs and jointly employing gaussian-weighted distribution to intensify semantic sequences. (3) Constructing a cross-modal graph to preserve the structured information of both image and text data and share information between modalities, while introducing sentiment knowledge to refine the edge weights of the graph to capture cross-modal sentiment interaction. We evaluate MDPF on three publicly available datasets across three tasks, and the empirical results demonstrate the superiority of our proposed model.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102747"},"PeriodicalIF":14.7,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142554835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Vul-LMGNNs: Fusing language models and online-distilled graph neural networks for code vulnerability detection","authors":"Ruitong Liu ,&nbsp;Yanbin Wang ,&nbsp;Haitao Xu ,&nbsp;Jianguo Sun ,&nbsp;Fan Zhang ,&nbsp;Peiyue Li ,&nbsp;Zhenhao Guo","doi":"10.1016/j.inffus.2024.102748","DOIUrl":"10.1016/j.inffus.2024.102748","url":null,"abstract":"<div><div>Code Language Models (codeLMs) and Graph Neural Networks (GNNs) are widely used in code vulnerability detection. However, a critical yet often overlooked issue is that GNNs primarily rely on aggregating information from adjacent nodes, limiting structural information transfer to single-layer updates. In code graphs, nodes and relationships typically require cross-layer information propagation to fully capture complex program logic and potential vulnerability patterns. Furthermore, while some studies utilize codeLMs to supplement GNNs with code semantic information, existing integration methods have not fully explored the potential of their collaborative effects.</div><div>To address these challenges, we introduce Vul-LMGNNs that integrates pre-trained CodeLMs with GNNs, leveraging knowledge distillation to facilitate cross-layer propagation of both code semantic knowledge and structural information. Specifically, Vul-LMGNNs utilizes Code Property Graphs (CPGs) to incorporate code syntax, control flow, and data dependencies, while employing gated GNNs to extract structural information in the CPG. To achieve cross-layer information transmission, we implement an online knowledge distillation (KD) program that enables a single student GNN to acquire structural information extracted from a simultaneously trained counterpart through an alternating training procedure. Additionally, we leverage pre-trained CodeLMs to extract semantic features from code sequences. Finally, we propose an ”implicit-explicit” joint training framework to better leverage the strengths of both CodeLMs and GNNs. In the implicit phase, we utilize CodeLMs to initialize the node embeddings of each student GNN. Through online knowledge distillation, we facilitate the propagation of both code semantics and structural information across layers. In the explicit phase, we perform linear interpolation between the CodeLM and the distilled GNN to learn a late fusion model. The proposed method, evaluated across four real-world vulnerability datasets, demonstrated superior performance compared to 17 state-of-the-art approaches. Our source code can be accessed via GitHub: <span><span>https://github.com/Vul-LMGNN/vul-LMGGNN</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102748"},"PeriodicalIF":14.7,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531903","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Dynamic clustering-based consensus model for large-scale group decision-making considering overlapping communities","authors":"Zhen Hua ,&nbsp;Xiangjie Gou ,&nbsp;Luis Martínez","doi":"10.1016/j.inffus.2024.102743","DOIUrl":"10.1016/j.inffus.2024.102743","url":null,"abstract":"<div><div>Consensus-reaching strategy is crucial in large-scale group decision-making (LSGDM) as it serves as an effective approach to reducing group conflicts. Meanwhile, the common social network relationships in large groups can affect information exchange, thereby influencing the consensus-reaching process (CRP) and decision results. Therefore, how to leverage social network information in LSGDM to obtain an agreed solution has received widespread attention. However, most existing research assumes relative independence between communities in the dimension reduction process of LSGDM and neglects the possibility of different overlaps between them. Moreover, the impact of overlapping communities on CRP has not been adequately explored. Besides, the dynamic variations in clusters and their weights caused by evaluation updates need to be further studied. To address these issues, this paper proposes a dynamic clustering-based consensus-reaching method for LSGDM considering the impact of overlapping communities. First, the LINE-based label propagation algorithm is designed to cluster decision makers (DMs) and detect overlapping communities with social network information. An overlapping community-driven feedback mechanism is then developed to enhance group consensus by utilizing the bridging role of overlapping DMs. During CRP, clusters and their weights are dynamically updated with trust evolution due to the evaluation iteration. Finally, a case study using the Film Trust dataset is conducted to verify the effectiveness of the proposed method. Simulation experiments and comparative analysis demonstrate the capability of our method in modeling practical scenarios and addressing LSGDM problems under social network contexts.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102743"},"PeriodicalIF":14.7,"publicationDate":"2024-10-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531904","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Applications of knowledge distillation in remote sensing: A survey","authors":"Yassine Himeur ,&nbsp;Nour Aburaed ,&nbsp;Omar Elharrouss ,&nbsp;Iraklis Varlamis ,&nbsp;Shadi Atalla ,&nbsp;Wathiq Mansoor ,&nbsp;Hussain Al-Ahmad","doi":"10.1016/j.inffus.2024.102742","DOIUrl":"10.1016/j.inffus.2024.102742","url":null,"abstract":"<div><div>With the ever-growing complexity of models in the field of remote sensing (RS), there is an increasing demand for solutions that balance model accuracy with computational efficiency. Knowledge distillation (KD) has emerged as a powerful tool to meet this need, enabling the transfer of knowledge from large, complex models to smaller, more efficient ones without significant loss in performance. This review article provides an extensive examination of KD and its innovative applications in RS. KD, a technique developed to transfer knowledge from a complex, often cumbersome model (teacher) to a more compact and efficient model (student), has seen significant evolution and application across various domains. Initially, we introduce the fundamental concepts and historical progression of KD methods. The advantages of employing KD are highlighted, particularly in terms of model compression, enhanced computational efficiency, and improved performance, which are pivotal for practical deployments in RS scenarios. The article provides a comprehensive taxonomy of KD techniques, where each category is critically analyzed to demonstrate the breadth and depth of the alternative options, and illustrates specific case studies that showcase the practical implementation of KD methods in RS tasks, such as instance segmentation and object detection. Further, the review discusses the challenges and limitations of KD in RS, including practical constraints and prospective future directions, providing a comprehensive overview for researchers and practitioners in the field of RS. Through this organization, the paper not only elucidates the current state of research in KD but also sets the stage for future research opportunities, thereby contributing significantly to both academic research and real-world applications.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102742"},"PeriodicalIF":14.7,"publicationDate":"2024-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142554838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Label distribution-driven multi-view representation learning","authors":"Wenbiao Yan ,&nbsp;Minghong Wu ,&nbsp;Yiyang Zhou ,&nbsp;Qinghai Zheng ,&nbsp;Jinqian Chen ,&nbsp;Haozhe Cheng ,&nbsp;Jihua Zhu","doi":"10.1016/j.inffus.2024.102727","DOIUrl":"10.1016/j.inffus.2024.102727","url":null,"abstract":"<div><div>In multi-view representation learning (MVRL), the challenge of category uncertainty is significant. Existing methods excel at deriving shared representations across multiple views, but often neglect the uncertainty associated with cluster assignments from each view, thereby leading to increased ambiguity in the category determination. Additionally, methods like kernel-based or neural network-based approaches, while revealing nonlinear relationships, lack attention to category uncertainty. To address these limitations, this paper proposes a method leveraging the uncertainty of label distributions to enhance MVRL. Specifically, our approach combines uncertainty reduction based on label distribution with view representation learning to improve clustering accuracy and robustness. It initially computes the within-view representation of the sample and semantic labels. Then, we introduce a novel constraint based on either variance or information entropy to mitigate class uncertainty, thereby improving the discriminative power of the learned representations. Extensive experiments conducted on diverse multi-view datasets demonstrate that our method consistently outperforms existing approaches, producing more accurate and reliable class assignments. The experimental results highlight the effectiveness of our method in enhancing MVRL by reducing category uncertainty and improving overall classification performance. This method is not only very interpretable but also enhances the model’s ability to learn multi-view consistent information.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102727"},"PeriodicalIF":14.7,"publicationDate":"2024-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142554834","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"HSMix: Hard and soft mixing data augmentation for medical image segmentation","authors":"D. Sun ,&nbsp;F. Dornaika ,&nbsp;N. Barrena","doi":"10.1016/j.inffus.2024.102741","DOIUrl":"10.1016/j.inffus.2024.102741","url":null,"abstract":"<div><div>Due to the high cost of annotation or the rarity of some diseases, medical image segmentation is often limited by data scarcity and the resulting overfitting problem. Self-supervised learning and semi-supervised learning can mitigate the data scarcity challenge to some extent. However, both of these paradigms are complex and require either hand-crafted pretexts or well-defined pseudo-labels. In contrast, data augmentation represents a relatively simple and straightforward approach to addressing data scarcity issues. It has led to significant improvements in image recognition tasks. However, the effectiveness of local image editing augmentation techniques in the context of segmentation has been less explored. Additionally, traditional data augmentation methods for local image editing augmentation methods generally utilize square regions, which cause a loss of contour information.</div><div>We propose HSMix, a novel approach to local image editing data augmentation involving hard and soft mixing for medical semantic segmentation. In our approach, a hard-augmented image is created by combining homogeneous regions (superpixels) from two source images. A soft mixing method further adjusts the brightness of these composed regions with brightness mixing based on locally aggregated pixel-wise saliency coefficients. The ground-truth segmentation masks of the two source images undergo the same mixing operations to generate the associated masks for the augmented images.</div><div>Our method fully exploits both the prior contour and saliency information, thus preserving local semantic information in the augmented images while enriching the augmentation space with more diversity. Our method is a plug-and-play solution that is model agnostic and applicable to a range of medical imaging modalities. Extensive experimental evidence has demonstrated its effectiveness in a variety of medical segmentation tasks. The source code is available in <span><span>https://github.com/DanielaPlusPlus/HSMix</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102741"},"PeriodicalIF":14.7,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531905","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An adaptive meta-imitation learning-based recommendation environment simulator: A case study on ship-cargo matching","authors":"Guangyao Pang ,&nbsp;Jiehang Xie ,&nbsp;Fei Hao","doi":"10.1016/j.inffus.2024.102740","DOIUrl":"10.1016/j.inffus.2024.102740","url":null,"abstract":"<div><div>High-quality shipping is one of the effective ways for sustainable cities in inland river basins to improve transportation efficiency and reduce energy consumption. Currently, the biggest challenge faced by shipping is the high empty-ship rate, which makes it impossible to directly apply machine learning methods due to the cold-start problem. Although some researchers have tried to utilize deep reinforcement learning(DRL)-based recommendation that do not rely on manually labeled data to alleviate the cold-start problem, progress has been slow due to the lack of available training environment. Therefore, this paper introduces an adaptive meta-imitation learning-based recommendation environment simulator, termed AMIL-Simulator. Specifically, we construct a conditionally guided diffusion model to simulate shipowner behavior in a dynamically changing environment. Moreover, we propose a shipowner reward model based on adaptive meta-imitation learning, enabling the learning of shipowner rewards across multiple tasks, even when confronted with limited samples and imbalanced categories. By conducting extensive quantitative experimental evaluations and shipowner-cargo matching studies, the results demonstrate the effectiveness of AMIL-Simulator, particularly in smaller-scale and cold-start environments.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102740"},"PeriodicalIF":14.7,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531896","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Recent advances in data-driven fusion of multi-modal imaging and genomics for precision medicine","authors":"Shuo Wang ,&nbsp;Meng Liu ,&nbsp;Yan Li ,&nbsp;Xinyu Zhang ,&nbsp;Mengting Sun ,&nbsp;Zian Wang ,&nbsp;Ruokun Li ,&nbsp;Qirong Li ,&nbsp;Qing Li ,&nbsp;Yili He ,&nbsp;Xumei Hu ,&nbsp;Longyu Sun ,&nbsp;Fuhua Yan ,&nbsp;Mengyao Yu ,&nbsp;Weiping Ding ,&nbsp;Chengyan Wang","doi":"10.1016/j.inffus.2024.102738","DOIUrl":"10.1016/j.inffus.2024.102738","url":null,"abstract":"<div><div>Imaging genomics is poised to revolutionize clinical practice by providing deep insights into the genetic underpinnings of disease, enabling early detection, and facilitating personalized treatment strategies. The field has seen remarkable advancements, with significant momentum fueled by cutting-edge imaging techniques, sophisticated data-driven fusion methods, and extensive large cohort datasets. Originally centered on the brain, imaging genomics has now expanded to encompass other organs throughout the body. Due to the highly interdisciplinary nature involving medical imaging, genetics, machine learning, and clinical medicine, readers who wish to conduct research in this field urgently need a comprehensive review. This survey provides an overview of recent advancements in data-driven fusion of multi-modal imaging and genomics, covering applications in the brain, heart, lungs, breasts, abdomen, and bones. We summarize three primary fusion strategies: correlation analysis, causal analysis, and machine learning, discussing their respective application scenarios. Additionally, we explore clinical applications that integrate imaging datasets and genomic data across six major organ systems, and present available open datasets featuring both modalities. Finally, we summarize the challenges and future directions in imaging genomics, which include improving data representation, integrating other omics data, conducting cross-dataset analyses, advancing machine learning algorithms, and investigating organ interactions. This survey aims to review the latest developments in data-driven fusion for precision medicine while providing insights into the future of this evolving field.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102738"},"PeriodicalIF":14.7,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531902","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An interactive iteration consensus based social network large-scale group decision making method and its application in zero-waste city evaluation","authors":"Fanyong Meng ,&nbsp;Hao Li ,&nbsp;Jinyu Li","doi":"10.1016/j.inffus.2024.102744","DOIUrl":"10.1016/j.inffus.2024.102744","url":null,"abstract":"<div><div>The construction of zero-waste (ZW) cities receives increasing attention from the Chinese government. The evaluation is essential to make policy variations according to the actual situation in each place. Previous assessments of ZW cities have primarily relied on historical data, which fails to account for the subjective preferences of various stakeholders. For example, it is challenging to capture residents' subjective opinions about the development of a ZW city. This paper presents a social network large-scale group decision-making method for evaluating the construction of ZW city. First, experts' evaluation opinions and trust relations are used to develop an improved clustering method. The weights of the clusters are then determined using internal-external cohesion indices and the number of experts, with experts' weights defined by their similarity-trust degree. An optimization model based on interactive iteration consensus is formulated, considering the fairness and rationality of allocation schemes. Additionally, a new social network large-scale group decision-making method is presented. Finally, the proposed method is illustrated with a case study of selecting a national-level ZW city in Jiangsu Province.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102744"},"PeriodicalIF":14.7,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}