An Asymptotic Multiscale Symmetric Fusion Network for Hyperspectral and Multispectral Image Fusion

IF 8.6 1区 地球科学 Q1 ENGINEERING, ELECTRICAL & ELECTRONIC
Shuaiqi Liu;Tingting Shao;Siyuan Liu;Bing Li;Yu-Dong Zhang
{"title":"An Asymptotic Multiscale Symmetric Fusion Network for Hyperspectral and Multispectral Image Fusion","authors":"Shuaiqi Liu;Tingting Shao;Siyuan Liu;Bing Li;Yu-Dong Zhang","doi":"10.1109/TGRS.2025.3525840","DOIUrl":null,"url":null,"abstract":"Despite the high spectral resolution and abundant information of hyperspectral images (HSIs), their spatial resolution is relatively low due to limitations in sensor technology. Sensors often need to sacrifice some spatial resolution to ensure accurate light energy measurement when pursuing high spectral resolution. This tradeoff results in HSI’s inability to capture fine spatial details, thereby limiting its application in scenarios requiring high-precision spatial information. HSI and multispectral image (MSI) fusion is a commonly used technique for generating high-resolution HSI (HR-HSI). However, many deep learning-based HSI-MSI fusion algorithms ignore correlation and multiscale information between input images. To address this issue, we propose an asymptotic multiscale symmetric fusion network (AMSF-Net) for hyperspectral and MSI fusion. AMSF-Net consists of two parts: the multilevel feature fusion (MFF) module and the progressive cross-scale spatial perception (PCP) module. The MFF module uses multistream feature extraction branches to perform information interaction between HSI and MSI at the same scale layer by layer, compensating for the spatial details lacking in HSI and the spectral details absent in MSI. The PCP module combines the input and output features of MFF, utilizes multiscale bidirectional strip convolution and deep convolution to further refine edge features, and reconstructs HR-HSI by learning the features of different expansion roll branches by connecting across scales. Comparative experiments with several state-of-the-art HSI-MSI fusion algorithms on four publicly available datasets, CAVE, Chikusei, Houston, and WorldView-3, are conducted to validate the effectiveness and superiority of AMSF-Net. On the Chikusei dataset, improvements were 9.1%, 12.5%, and 5.1%, respectively, on the indicators root-mean-square error (RMSE), error of relative global accuracy in synthesis (ERGAS), and spectral angle mapper (SAM), compared to the suboptimal method.","PeriodicalId":13213,"journal":{"name":"IEEE Transactions on Geoscience and Remote Sensing","volume":"63 ","pages":"1-16"},"PeriodicalIF":8.6000,"publicationDate":"2025-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Geoscience and Remote Sensing","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10824890/","RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

Abstract

Despite the high spectral resolution and abundant information of hyperspectral images (HSIs), their spatial resolution is relatively low due to limitations in sensor technology. Sensors often need to sacrifice some spatial resolution to ensure accurate light energy measurement when pursuing high spectral resolution. This tradeoff results in HSI’s inability to capture fine spatial details, thereby limiting its application in scenarios requiring high-precision spatial information. HSI and multispectral image (MSI) fusion is a commonly used technique for generating high-resolution HSI (HR-HSI). However, many deep learning-based HSI-MSI fusion algorithms ignore correlation and multiscale information between input images. To address this issue, we propose an asymptotic multiscale symmetric fusion network (AMSF-Net) for hyperspectral and MSI fusion. AMSF-Net consists of two parts: the multilevel feature fusion (MFF) module and the progressive cross-scale spatial perception (PCP) module. The MFF module uses multistream feature extraction branches to perform information interaction between HSI and MSI at the same scale layer by layer, compensating for the spatial details lacking in HSI and the spectral details absent in MSI. The PCP module combines the input and output features of MFF, utilizes multiscale bidirectional strip convolution and deep convolution to further refine edge features, and reconstructs HR-HSI by learning the features of different expansion roll branches by connecting across scales. Comparative experiments with several state-of-the-art HSI-MSI fusion algorithms on four publicly available datasets, CAVE, Chikusei, Houston, and WorldView-3, are conducted to validate the effectiveness and superiority of AMSF-Net. On the Chikusei dataset, improvements were 9.1%, 12.5%, and 5.1%, respectively, on the indicators root-mean-square error (RMSE), error of relative global accuracy in synthesis (ERGAS), and spectral angle mapper (SAM), compared to the suboptimal method.
高光谱与多光谱图像融合的渐近多尺度对称融合网络
高光谱图像虽然具有较高的光谱分辨率和丰富的信息,但由于传感器技术的限制,其空间分辨率相对较低。传感器在追求高光谱分辨率的同时,往往需要牺牲一定的空间分辨率来保证准确的光能测量。这种权衡导致HSI无法捕获精细的空间细节,从而限制了它在需要高精度空间信息的场景中的应用。HSI和多光谱图像(MSI)融合是生成高分辨率HSI (HR-HSI)的常用技术。然而,许多基于深度学习的HSI-MSI融合算法忽略了输入图像之间的相关性和多尺度信息。为了解决这个问题,我们提出了一个用于高光谱和MSI融合的渐近多尺度对称融合网络(AMSF-Net)。AMSF-Net由两部分组成:多层特征融合(MFF)模块和逐级跨尺度空间感知(PCP)模块。MFF模块利用多流特征提取分支,在同一尺度上逐层进行HSI和MSI之间的信息交互,补偿HSI中缺失的空间细节和MSI中缺失的光谱细节。PCP模块结合MFF的输入输出特征,利用多尺度双向条形卷积和深度卷积进一步细化边缘特征,通过跨尺度连接,学习不同扩展卷分支的特征,重建HR-HSI。在CAVE、Chikusei、Houston和WorldView-3四个公开数据集上与几种最先进的HSI-MSI融合算法进行了对比实验,验证了AMSF-Net的有效性和优越性。在Chikusei数据集上,与次优方法相比,该方法在指标均方根误差(RMSE)、相对全局合成精度误差(ERGAS)和光谱角映射器(SAM)上分别提高了9.1%、12.5%和5.1%。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
IEEE Transactions on Geoscience and Remote Sensing
IEEE Transactions on Geoscience and Remote Sensing 工程技术-地球化学与地球物理
CiteScore
11.50
自引率
28.00%
发文量
1912
审稿时长
4.0 months
期刊介绍: IEEE Transactions on Geoscience and Remote Sensing (TGRS) is a monthly publication that focuses on the theory, concepts, and techniques of science and engineering as applied to sensing the land, oceans, atmosphere, and space; and the processing, interpretation, and dissemination of this information.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信