SSSAT-Net: Spectral-Spatial Self-Attention-Based Transformer Network for hyperspectral image classification

Abstract

In recent years, Convolutional Neural Networks (CNNs) have demonstrated remarkable performance in hyperspectral image classification. However, due to the high-dimensional redundancy of hyperspectral data, different spectral bands contribute significantly varying relevance to classification tasks. Additionally, existing networks commonly adopt patch-based input strategies, which struggle to effectively model spatial correlations between central pixels and their neighboring regions. The local receptive fields inherent to traditional CNNs further limit their ability to capture global features. To address these challenges, this study proposes a Spectral-Spatial Self-Attention-Based Transformer Network (SSSAT). The model first employs Principal Component Analysis (PCA) to reduce dimensionality, followed by the Convolutional Block Attention Module (CBAM) to selectively extract discriminative spectral-spatial features. Subsequently, the Spectral Attention Module (SpeAM) integrates Squeeze-and-Excitation (SE) attention with depthwise separable convolution to achieve adaptive calibration of spectral bands. The Spatial Attention Module (SpaAM) is further constructed to enhance spatial feature representation through a Manhattan Distance based Feature Vector Self-similarity Attention module (MDSA), while the Multi-Scale Convolutional Information Fusion module (MSCIF) explores spatial characteristics at multiple scales. Additionally, the Transformer architecture is utilized to extract global spectral-spatial features. Finally, classification is performed via a linear layer. Experimental results on five public datasets (Indian Pines, Kennedy Space Center, Pavia University, Houston 2013 and Salinas) demonstrate that SSSAT achieves superior classification performance compared to state-of-the-art methods.