Dual-Path Interactive U-Net for Unsupervised Hyperspectral Image Super-Resolution

Abstract

Combining low-spatial-resolution hyperspectral image (LrHSI) with high-spatial-resolution multispectral image (HrMSI) serves as an effective strategy for enhancing the spatial fidelity of LrHSI. Nevertheless, most existing methods still face challenges in effectively leveraging the complementary information between the two distinct modalities and maintaining internal consistency, leading to suboptimal fusion results. Previous studies have demonstrated that U-shaped networks can capture spatial structural features within images. Inspired by this, we propose a dual-path interactive U-Net architecture to preserve spatial and spectral integrity. Specifically, we use a standard U-Net and a reversed U-Net as the backbone to extract image information and generate abundance maps of the input images. By enabling interaction between the encoders and decoders of both U-Nets, our architecture integrates information across different scales and modes, leading to enhanced fusion results. To further improve the feature extraction capability, we construct a multimode decomposition and reconstruction module, which adaptively fuses the features of LrHSI and HrMSI. This module extracts and combines correlations between the images through canonical polyadic decomposition and attention mechanism, capturing global features across different modes. In addition, we design a weight-sharing U-Net that leverages the similarities and differences between two abundance maps, ensuring internal consistency while reducing computational cost. Thorough evaluations conducted using four publicly available datasets, along with one real-world dataset, and under various noise conditions confirm the validity of our proposed model.