{"title":"红外与可见光图像融合的多级自适应注意力融合网络","authors":"Ziming Hu;Quan Kong;Qing Liao","doi":"10.1109/LSP.2024.3509341","DOIUrl":null,"url":null,"abstract":"Infrared and visible image fusion involves integrating complementary or critical information extracted from different source images into one image. Due to the significant differences between the two modality features and those across different scales, traditional fusion strategies, such as addition or concatenation, often result in information redundancy or the degradation of crucial information. This letter proposes a multi-level adaptive attention fusion network to adaptively fuse features extracted from different sources. Specifically, we introduced an Adaptive Scale Attention Fusion (ASAF) module that uses a soft selection mechanism to assess the relative importance of different modality features at the same scale and assign corresponding fusion weights. Additionally, a guided upsampling layer is utilized to integrate shallow and deep feature information at different scales in the multi-scale structure. Qualitative and quantitative results on public datasets validate the superior performance of our approach in both visual effects and quantitative metrics.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"366-370"},"PeriodicalIF":3.2000,"publicationDate":"2024-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Multi-Level Adaptive Attention Fusion Network for Infrared and Visible Image Fusion\",\"authors\":\"Ziming Hu;Quan Kong;Qing Liao\",\"doi\":\"10.1109/LSP.2024.3509341\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Infrared and visible image fusion involves integrating complementary or critical information extracted from different source images into one image. Due to the significant differences between the two modality features and those across different scales, traditional fusion strategies, such as addition or concatenation, often result in information redundancy or the degradation of crucial information. This letter proposes a multi-level adaptive attention fusion network to adaptively fuse features extracted from different sources. Specifically, we introduced an Adaptive Scale Attention Fusion (ASAF) module that uses a soft selection mechanism to assess the relative importance of different modality features at the same scale and assign corresponding fusion weights. Additionally, a guided upsampling layer is utilized to integrate shallow and deep feature information at different scales in the multi-scale structure. Qualitative and quantitative results on public datasets validate the superior performance of our approach in both visual effects and quantitative metrics.\",\"PeriodicalId\":13154,\"journal\":{\"name\":\"IEEE Signal Processing Letters\",\"volume\":\"32 \",\"pages\":\"366-370\"},\"PeriodicalIF\":3.2000,\"publicationDate\":\"2024-11-28\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Signal Processing Letters\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10771638/\",\"RegionNum\":2,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Signal Processing Letters","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10771638/","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
Multi-Level Adaptive Attention Fusion Network for Infrared and Visible Image Fusion
Infrared and visible image fusion involves integrating complementary or critical information extracted from different source images into one image. Due to the significant differences between the two modality features and those across different scales, traditional fusion strategies, such as addition or concatenation, often result in information redundancy or the degradation of crucial information. This letter proposes a multi-level adaptive attention fusion network to adaptively fuse features extracted from different sources. Specifically, we introduced an Adaptive Scale Attention Fusion (ASAF) module that uses a soft selection mechanism to assess the relative importance of different modality features at the same scale and assign corresponding fusion weights. Additionally, a guided upsampling layer is utilized to integrate shallow and deep feature information at different scales in the multi-scale structure. Qualitative and quantitative results on public datasets validate the superior performance of our approach in both visual effects and quantitative metrics.
期刊介绍:
The IEEE Signal Processing Letters is a monthly, archival publication designed to provide rapid dissemination of original, cutting-edge ideas and timely, significant contributions in signal, image, speech, language and audio processing. Papers published in the Letters can be presented within one year of their appearance in signal processing conferences such as ICASSP, GlobalSIP and ICIP, and also in several workshop organized by the Signal Processing Society.