{"title":"An Asymptotic Multiscale Symmetric Fusion Network for Hyperspectral and Multispectral Image Fusion","authors":"Shuaiqi Liu;Tingting Shao;Siyuan Liu;Bing Li;Yu-Dong Zhang","doi":"10.1109/TGRS.2025.3525840","DOIUrl":null,"url":null,"abstract":"Despite the high spectral resolution and abundant information of hyperspectral images (HSIs), their spatial resolution is relatively low due to limitations in sensor technology. Sensors often need to sacrifice some spatial resolution to ensure accurate light energy measurement when pursuing high spectral resolution. This tradeoff results in HSI’s inability to capture fine spatial details, thereby limiting its application in scenarios requiring high-precision spatial information. HSI and multispectral image (MSI) fusion is a commonly used technique for generating high-resolution HSI (HR-HSI). However, many deep learning-based HSI-MSI fusion algorithms ignore correlation and multiscale information between input images. To address this issue, we propose an asymptotic multiscale symmetric fusion network (AMSF-Net) for hyperspectral and MSI fusion. AMSF-Net consists of two parts: the multilevel feature fusion (MFF) module and the progressive cross-scale spatial perception (PCP) module. The MFF module uses multistream feature extraction branches to perform information interaction between HSI and MSI at the same scale layer by layer, compensating for the spatial details lacking in HSI and the spectral details absent in MSI. The PCP module combines the input and output features of MFF, utilizes multiscale bidirectional strip convolution and deep convolution to further refine edge features, and reconstructs HR-HSI by learning the features of different expansion roll branches by connecting across scales. Comparative experiments with several state-of-the-art HSI-MSI fusion algorithms on four publicly available datasets, CAVE, Chikusei, Houston, and WorldView-3, are conducted to validate the effectiveness and superiority of AMSF-Net. On the Chikusei dataset, improvements were 9.1%, 12.5%, and 5.1%, respectively, on the indicators root-mean-square error (RMSE), error of relative global accuracy in synthesis (ERGAS), and spectral angle mapper (SAM), compared to the suboptimal method.","PeriodicalId":13213,"journal":{"name":"IEEE Transactions on Geoscience and Remote Sensing","volume":"63 ","pages":"1-16"},"PeriodicalIF":8.6000,"publicationDate":"2025-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Geoscience and Remote Sensing","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10824890/","RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0
Abstract
Despite the high spectral resolution and abundant information of hyperspectral images (HSIs), their spatial resolution is relatively low due to limitations in sensor technology. Sensors often need to sacrifice some spatial resolution to ensure accurate light energy measurement when pursuing high spectral resolution. This tradeoff results in HSI’s inability to capture fine spatial details, thereby limiting its application in scenarios requiring high-precision spatial information. HSI and multispectral image (MSI) fusion is a commonly used technique for generating high-resolution HSI (HR-HSI). However, many deep learning-based HSI-MSI fusion algorithms ignore correlation and multiscale information between input images. To address this issue, we propose an asymptotic multiscale symmetric fusion network (AMSF-Net) for hyperspectral and MSI fusion. AMSF-Net consists of two parts: the multilevel feature fusion (MFF) module and the progressive cross-scale spatial perception (PCP) module. The MFF module uses multistream feature extraction branches to perform information interaction between HSI and MSI at the same scale layer by layer, compensating for the spatial details lacking in HSI and the spectral details absent in MSI. The PCP module combines the input and output features of MFF, utilizes multiscale bidirectional strip convolution and deep convolution to further refine edge features, and reconstructs HR-HSI by learning the features of different expansion roll branches by connecting across scales. Comparative experiments with several state-of-the-art HSI-MSI fusion algorithms on four publicly available datasets, CAVE, Chikusei, Houston, and WorldView-3, are conducted to validate the effectiveness and superiority of AMSF-Net. On the Chikusei dataset, improvements were 9.1%, 12.5%, and 5.1%, respectively, on the indicators root-mean-square error (RMSE), error of relative global accuracy in synthesis (ERGAS), and spectral angle mapper (SAM), compared to the suboptimal method.
期刊介绍:
IEEE Transactions on Geoscience and Remote Sensing (TGRS) is a monthly publication that focuses on the theory, concepts, and techniques of science and engineering as applied to sensing the land, oceans, atmosphere, and space; and the processing, interpretation, and dissemination of this information.