Information Fusion最新文献

{"title":"Stimulating conversation-style emergencies of multi-modal LMs","authors":"Shun Qian ,&nbsp;Bingquan Liu ,&nbsp;Chengjie Sun ,&nbsp;Zhen Xu ,&nbsp;Baoxun Wang","doi":"10.1016/j.inffus.2025.103047","DOIUrl":"10.1016/j.inffus.2025.103047","url":null,"abstract":"<div><div>The multi-modal Language Models (LMs) perform very well on alignment-style tasks such as Image–Text Retrieval and Image Captioning, benefiting mainly from pre-training on numerous image–text pairs. However, our evaluations indicate that these models underperform on conversation-style multi-modal tasks, such as Image-Chat and Visual Dialog, which constitute a crucial segment of multi-modal applications. To bridge this gap, this paper proposes a novel pre-training task, named as MBCG, to stimulate the abilities of existing multi-modal LMs on conversation-style multi-modal tasks without hurting their intrinsic abilities. For this purpose, we collect two image–text-comments triplet multi-modal datasets in both English and Chinese to apply the new pre-training task to existing models. The experimental results reveal that the MBCG task can significantly boost the performance of these models on conversation-style tasks, without any noticeable performance decline on their original evaluation tasks.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"120 ","pages":"Article 103047"},"PeriodicalIF":14.7,"publicationDate":"2025-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143550295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multi-fidelity modeling method based on adaptive transfer learning","authors":"Fazhi Tang ,&nbsp;Yubo Li ,&nbsp;Jun Huang ,&nbsp;Feng Liu","doi":"10.1016/j.inffus.2025.103045","DOIUrl":"10.1016/j.inffus.2025.103045","url":null,"abstract":"<div><div>A wide range of problems in engineering applications require high-fidelity data to reflect real physical phenomena. However, only a small amount of high-fidelity data can be maintained under data acquisition cost constraints, which makes it difficult to meet the high-fidelity data needs of engineering application scenarios. This paper presents a multi-fidelity transfer modeling method, namely the Adaptive Transfer Learning Net (AtNet), aiming to suppress the overfitting phenomenon caused by the scarcity of high-fidelity data samples in multi-source data fusion. This method constructs multiple intermediate domains similar to the characteristics of the high-fidelity domain based on the minimum transfer loss theory, divides the transfer task into multi-stage transfer processes, and introduces rich and diverse feature information for the model to suppress the overfitting phenomenon. Finally, AtNet is applied to the multi-fidelity modeling task of numerical examples. Compared with traditional multi-fidelity modeling methods, AtNet not only maintains a high prediction accuracy but also can still effectively reflect the laws of high-fidelity data after further reducing the high-fidelity training data. In particular, AtNet is applied to multifidelity modeling scenarios involving SG6043 airfoil aerodynamics data and ONERA M6 pressure distribution data, comparative experiments were conducted by reducing high-fidelity training data. The results demonstrate that AtNet can approximate the true physical laws of aerodynamics using minimal high-fidelity training data, thereby reducing the cost of high-precision aerodynamic parameter design.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"120 ","pages":"Article 103045"},"PeriodicalIF":14.7,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143562525","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multi-stage multimodal fusion network with language models and uncertainty evaluation for early risk stratification in rheumatic and musculoskeletal diseases","authors":"Bing Wang ,&nbsp;Weizi Li ,&nbsp;Anthony Bradlow ,&nbsp;Archie Watt ,&nbsp;Antoni T.Y. Chan ,&nbsp;Eghosa Bazuaye","doi":"10.1016/j.inffus.2025.103068","DOIUrl":"10.1016/j.inffus.2025.103068","url":null,"abstract":"<div><div>Precise risk stratification of rheumatic musculoskeletal diseases (RMDs) is crucial for ensuring patients get right referrals and treatments quickly. However, it is challenging due to the non-specific symptoms and the lack of the diagnostically definitive single biomarker. The real-world referral data present several challenges such as the free format texts and incomplete data challenges, which introduces further modeling complexity, and makes uncertainty quantification crucial for ensuring reliable predictions and outcomes. To solve these challenges, we developed a multi-stage multimodal fusion network with conformal prediction method that can accurately risk stratify RMDs at the point of referrals, quantify the uncertainty and flag unreliable predictions for physician's interventions. The proposed models were trained and evaluated using referral data from 128 General Practices (GPs) in the UK, which include patients who visited and were referred by GPs with suspected inflammatory conditions in RMDs between February 2018 and January 2024. Our model achieved 0.73 accuracy, 0.79 AUC, and 0.75 G-Mean to differentiate inflammatory conditions (IC) and non-inflammatory conditions (NIC) using patients’ presenting condition description (PCD) and medical history (MH) data, and 0.90 accuracy, 0.92 AUC, and 0.89 G-Mean using patients’ PCD, MH and additional blood test data (BTD). Furthermore, conformal prediction-based method has been developed to evaluate prediction uncertainty and can further identify 75.71 % unreliable predictions for patients with PCD and MH data, and 66.67 % unreliable predictions for patients with additional BTD data, which could be given a second-round examination by GP/secondary care clinicians for patient safety. The findings of this study suggest that language models with multi-stage multimodal fusion and uncertainty evaluation can risk stratify RMDs accurately using data available at the point of referral in the real world. Therefore, it is possible to be used by GPs and clinicians to help patients get the right treatment faster, demonstrating practical potential to improve RMDs referrals in the real world.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"120 ","pages":"Article 103068"},"PeriodicalIF":14.7,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143610820","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Unlocking the potential of quantum computing in prefabricated construction supply chains: Current trends, challenges, and future directions","authors":"Zhen-Song Chen ,&nbsp;Yue Tan ,&nbsp;Zheng Ma ,&nbsp;Zhengze Zhu ,&nbsp;Mirosław J. Skibniewski","doi":"10.1016/j.inffus.2025.103043","DOIUrl":"10.1016/j.inffus.2025.103043","url":null,"abstract":"<div><div>The intricate nature of prefabricated construction supply chain management (PCSCM) presents ongoing challenges in production scheduling, inventory control, and logistics coordination. Recent advances in quantum computing (QC) offer compelling approaches to address these multifaceted issues by enabling significantly faster and more precise optimization. This paper systematically reviews and synthesizes existing QC research in the supply chain context, particularly focusing on quantum algorithms that target the PCSCM lifecycle. Our analysis identifies three key domains: production, inventory, and transportation, in which QC can outperform classical methods, as evidenced by enhanced scheduling flexibility and cost minimization. However, our findings also highlight crucial bottlenecks, including quantum hardware limitations, organizational readiness gaps, and a lack of specialized interdisciplinary talent. We propose a framework of strategies to guide QC adoption, such as specialized algorithm development, collaborative research partnerships, and standardized data protocols. These insights offer promising future directions for leveraging QC to streamline operations and boost sustainability in the prefabricated construction sector.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"120 ","pages":"Article 103043"},"PeriodicalIF":14.7,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143579944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"PASAL: Progress- and sparsity-aware loss balancing for heterogeneous dataset fusion","authors":"Thomas Gorges ,&nbsp;Teresa Scholz ,&nbsp;Stefan Saloman ,&nbsp;Mathias Zinnen ,&nbsp;Juliane Hoffmann ,&nbsp;Nora Gourmelon ,&nbsp;Andreas Maier ,&nbsp;Sebastian Hettenkofer ,&nbsp;Vincent Christlein","doi":"10.1016/j.inffus.2025.103038","DOIUrl":"10.1016/j.inffus.2025.103038","url":null,"abstract":"<div><div>Machine learning has seen widespread application in many areas. Despite theoretical advancements, the demand for qualitative and extensive data foundations is increasing. Real-world datasets are often small and combining them is challenging due to the resulting sparsity and heterogeneity. Existing combination techniques merge datasets into a common space before training, causing drawbacks such as data loss and distortion of annotations. To address this, we fuse heterogeneous datasets by jointly training dataset-specific weighted sub-networks. Balancing losses from heterogeneous data sources is challenging, as current techniques are inadequate. We propose a novel progress- and sparsity-aware loss balancing method (PASAL), which adaptively balances sub-network losses based on individual learning progress and sparsity. As an example, we present the application of PASAL to the olfaction domain, where predicting smell properties based on molecular structure is difficult due to subjective impressions, typically limited data, and a lack of unified datasets. By evaluating PASAL on the DREAM Olfaction Prediction Challenge, we improve the current state-of-the-art method from a Z-Score of 9.92 to 10.10. Furthermore, by treating our AI as an annotator, we surpass human performance in the odor and pleasantness categories with statistical significance. Our findings are supported by a feature analysis, indicating that our heterogeneous combination methodology enhances odor prediction.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"120 ","pages":"Article 103038"},"PeriodicalIF":14.7,"publicationDate":"2025-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143593045","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"PEAFusion: Parameter-efficient Adaptation for RGB-Thermal fusion-based semantic segmentation","authors":"Yan Wang ,&nbsp;Henry K. Chu ,&nbsp;Yuxiang Sun","doi":"10.1016/j.inffus.2025.103030","DOIUrl":"10.1016/j.inffus.2025.103030","url":null,"abstract":"<div><div>RGB-Thermal (RGB-T) semantic segmentation has attracted great attention in the research community of autonomous driving. Full fine-tuning pre-trained networks is a common strategy in RGB-T semantic segmentation. However, as model size grows, updating all parameters becomes expensive and impractical, which hinders the wide applications of pre-trained networks despite their effectiveness. To efficiently adapt pre-trained single-modality networks to the multi-modal RGB-T task, we design a module named multi-view adapter-pair. The multi-view adapter-pair bridges the gap between pre-trained features and the features required for RGB-T semantic segmentation. It achieves this by approximating high-dimensional updates to the hidden state during full fine-tuning within low-dimensional spaces. Moreover, we propose cross-modal self-attention, constructed using the self-attention operations in pre-trained transformer models. The cross-modal self-attention is designed to fuse RGB and thermal data by expanding the self-attention mechanism in the pre-trained model from a single modality to multiple modalities. Due to the permutation invariance of the attention mechanism and the differences between the two modalities, we introduce modality bias to guide the attention mechanism in learning dependencies inter- and intra-the two modalities. Leveraging these innovations, our network outperforms state-of-the-art methods on the MFNet dataset, as well as the FMB dataset and PST900 dataset, while maintaining parameter efficiency.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"120 ","pages":"Article 103030"},"PeriodicalIF":14.7,"publicationDate":"2025-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143601457","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A multi-step interaction network for multi-class classification based on OCT and OCTA images","authors":"Han Zhang,&nbsp;Xuening Bai,&nbsp;Guangyao Hou,&nbsp;Xiongwen Quan","doi":"10.1016/j.inffus.2025.103041","DOIUrl":"10.1016/j.inffus.2025.103041","url":null,"abstract":"<div><div>OCT and OCTA images are important basis for diagnosing multiple ophthalmic diseases. However, it is a challenge to fuse these two modalities with high redundancy and simultaneous projection of 3D data for multi-class classification. This paper proposes a novel Multi-step Interaction Network (MINet) for projection and feature fusion as a unified framework, where OCT and OCTA images deeply interact for a seven-class classification task of ophthalmic diseases. Firstly, we design a Multi-modal Interaction Projection Module to iterate projection and shallow information interaction for effective feature selection. Secondly, a Feature Redundancy Removal Module compares the feature difference information between the two modalities to eliminate redundancy. Thirdly, the Feature Interaction Fusion Module utilizes the differential modal information from the backbone CNN to perform respective modal attention and achieve interactive fusion. Finally, a classifier module generates multi-class classification results. Experimental results show that our method achieved Accuracy of 0.8690, Precision of 0.6921, Recall of 0.7250, and F1 score of 0.7081 on the OCTA-500 dataset. Comparative experiments with other state-of-the-art methods for OCT image classification, along with ablation experiments, demonstrate the superior performance of MINet.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"120 ","pages":"Article 103041"},"PeriodicalIF":14.7,"publicationDate":"2025-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143550294","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"IFIFusion: A independent feature information fusion model for surface defect detection","authors":"Xin Zhou ,&nbsp;Yongchao Zhang ,&nbsp;Zheng Liu ,&nbsp;Zeyu Jiang ,&nbsp;Zhaohui Ren ,&nbsp;Tianchuan Mi ,&nbsp;Shihua Zhou","doi":"10.1016/j.inffus.2025.103039","DOIUrl":"10.1016/j.inffus.2025.103039","url":null,"abstract":"<div><div>Existing surface-defect detection networks often rely on pre-trained classification models as a backbone to extract multi-scale features and establish decoded modules tailored for these features. However, since the classification model primarily captures identifiable class features in natural images, these decoded modules struggle to accurately delineate the precise spatial structures from the multi-scale feature maps of the backbone. Also, discrete feature contents from different modules lack the necessary fusion, and the architecture lacks the ability to accurately classify the defective object in images, resulting in lower detection performance. Therefore, this paper explored an Independent Feature Information Fusion model (IFIFusion) consisting of spatial multi-axis information fusion, independent classification &amp; spatial information fusion, and multi-scale features fusion for defect detection tasks. This model fully explores the collaborative relationship between independent modules. Firstly, this work designs a Dual-Attention transformer backbone to extract classification features and a Light-Weight multi-scale Network (LWNet) parallel to the backbone to provide independent features for the proposed Spatial multi-Axis Fusion module (SAF). This study provides an independent feature fusion module (IFF), primarily consisting of the connector, convolution layers, batch normalization, ReLU, and interpolation method, to enrich discrete feature contents. Then, the SAF aggregates and fuses the discriminative spatial features from non-classification feature maps. Lastly, the paper fuses the feature maps at each scale to achieve effective coupling and interaction, enhancing the presentation of defective features. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art algorithms regarding visual quality and quantitative evaluations. The code is available at <span><span>https://github.com/zhx-hub/IFIFusion/tree/main</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"120 ","pages":"Article 103039"},"PeriodicalIF":14.7,"publicationDate":"2025-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143550413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Event-triggered set-membership fusion estimation of multi-rate multi-sensor systems under multiple cyber attacks","authors":"Zihao Cai ,&nbsp;Zehui Xiao ,&nbsp;Ming Lin ,&nbsp;Zheqing Zhou ,&nbsp;Jie Tao","doi":"10.1016/j.inffus.2025.103040","DOIUrl":"10.1016/j.inffus.2025.103040","url":null,"abstract":"<div><div>The article concerns the multi-rate multi-sensor systems set-membership fusion estimation problem under multiple cyber attacks. In order to save limited communication resources, a novel adaptive event-triggered strategy is developed to control the frequency of information transmission. In contrast to conventional adaptive strategies, a transformation law is introduced to establish the triggering condition, thereby effectively avoiding triggering behaviors caused by small fluctuations after the error has converged. In addition, a hybrid approach is proposed to address the issue of rate inconsistency among various components, significantly improving both the efficiency and accuracy of the estimation algorithm. Then, a cryptography-based privacy protection scheme is presented to defend against deception attacks and replay attacks. By incorporating the concept of set-membership estimation, an optimization problem for secure fusion estimation is formulated. In light of this, an online recursive algorithm is proposed to continuously obtain the weighting coefficients and the optimal ellipsoid set, ensuring that the error remains confined within the desired ellipsoid. Finally, the superiority and feasibility of the proposed scheme are validated through a simulation example and a practical experiment.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"120 ","pages":"Article 103040"},"PeriodicalIF":14.7,"publicationDate":"2025-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143550300","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enhancing trust in automated 3D point cloud data interpretation through explainable counterfactuals","authors":"Andreas Holzinger ,&nbsp;Niko Lukač ,&nbsp;Dzemail Rozajac ,&nbsp;Emile Johnston ,&nbsp;Veljka Kocic ,&nbsp;Bernhard Hoerl ,&nbsp;Christoph Gollob ,&nbsp;Arne Nothdurft ,&nbsp;Karl Stampfer ,&nbsp;Stefan Schweng ,&nbsp;Javier Del Ser","doi":"10.1016/j.inffus.2025.103032","DOIUrl":"10.1016/j.inffus.2025.103032","url":null,"abstract":"<div><div>This paper introduces a novel framework for augmenting explainability in the interpretation of point cloud data by fusing expert knowledge with counterfactual reasoning. Given the complexity and voluminous nature of point cloud datasets, derived predominantly from LiDAR and 3D scanning technologies, achieving interpretability remains a significant challenge, particularly in smart cities, smart agriculture, and smart forestry. This research posits that integrating expert knowledge with counterfactual explanations – speculative scenarios illustrating how altering input data points could lead to different outcomes – can significantly reduce the opacity of deep learning models processing point cloud data. The proposed optimization-driven framework utilizes expert-informed ad-hoc perturbation techniques to generate meaningful counterfactual scenarios when employing state-of-the-art deep learning architectures. The optimization process minimizes a multi-criteria objective comprising counterfactual metrics such as similarity, validity, and sparsity, which are specifically tailored for point cloud datasets. These metrics provide a quantitative lens for evaluating the interpretability of the counterfactuals. Furthermore, the proposed framework allows for the definition of explicit interpretable counterfactual perturbations at its core, thereby involving the audience of the model in the counterfactual generation pipeline and ultimately, improving their overall trust in the process. Results demonstrate a notable improvement in both the interpretability of the model’s decisions and the actionable insights delivered to end-users. Additionally, the study explores the role of counterfactual reasoning, coupled with expert input, in enhancing trustworthiness and enabling human-in-the-loop decision-making processes. By bridging the gap between complex data interpretations and user comprehension, this research advances the field of explainable AI, contributing to the development of transparent, accountable, and human-centered artificial intelligence systems.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"119 ","pages":"Article 103032"},"PeriodicalIF":14.7,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143508890","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}