PMA²-Net: Progressive Fusion of Multiscale Axial Attention Network for Hyperspectral and Multispectral Images

Abstract

Integrating low-resolution hyperspectral images (LR-HSIs) with corresponding high-resolution multispectral images (HR-MSIs) for the reconstruction of HR-HSI using deep learning techniques represents a critical area of research. Although convolutional neural networks (CNNs) are widely used for HR-HSI reconstruction, their small receptive fields hinder effective global feature extraction, which limits their potential. Fusion methods that rely on traditional attention mechanisms also lack feature interaction, failing to integrate and harmonize feature information from hyperspectral image (HSI) and MSI effectively. To address these issues, this article develops a novel progressive fusion of multiscale axial attention network (PMA2-Net), which combines multiscale convolutions with axial attention (AA) and employs a progressive interaction approach to reconstruct high-resolution images. Specifically, PMA2-Net extracts the spatial and spectral information from HSI through spatial feature extraction (Spatial-FE) and spectral feature extraction (Spectral-FE). Concurrently, a feature injection module (FIM) is introduced, employing multiscale convolution to capture local features and integrating AA to enhance global feature association. Moreover, a progressive fusion module (PFM) is employed to enhance multidimensional feature collaboration and hierarchical integration. Extensive studies conducted using five significant HSI datasets verify the effectiveness of PMA2-Net, demonstrating its superior performance compared to current state-of-the-art (SOTA) fusion techniques.