Hyperspectral image classification using local-to-global retention network

Abstract

Several transformer-based methods have demonstrated competitiveness in the hyperspectral image (HSI) classification tasks. However, the transformer has an expensive and slow inference due to the computational complexity of the self-attention mechanism, which increases the risk of a bottleneck while processing all the tokens for high-resolution images. Retentive Network (RetNet) has recently been proposed for large language models to achieve training parallelism, low-cost inference, and high performance. However, RetNet also struggles to capture local information, such as conventional transformers. This study proposes local-to-global retention (LGRetention) to overcome the issues. The proposed method directly obtains the inherent features from the original spatial-spectral patch, thereby preserving the spatial distribution and intrinsic spectral-spatial correlation. Moreover, the proposed LGRetention includes a local-to-global self-retention (LGSR), which contains Local Convolutional Network (LCN) and Global Self-Retention (GSR) modules created using three-dimensional (3D) convolutional neural networks (CNNs) and a developed HSI Retention mechanism to capture local and global features and enhance the long-range dependencies and spectral-spatial feature learning ability, thereby capturing both fine and coarse-grained data. In the HSI Retention mechanism, we developed a 3D parallel retention mechanism to retain the degradation between succeeding horizontal and vertical positions with the spectral dimension of the HSI data. Experimental results on three benchmark HSI datasets demonstrate that the proposed LGRetention outperforms the state-of-the-art methods. It increases accuracy and creates finer classification maps, demonstrating robustness and generalizability in the HSI classification task.