Retentive Spatial–Spectral Transformer for hyperspectral image denoising

Retention mechanism has emerged as a promising variant of Transformer and achieved remarkable success in natural language processing and computer vision. However, existing vision retention mechanism only explores spatial prior, suffering from limited representation for cubic spatial–spectral feature of hyperspectral image (HSI). To tackle this issue, we propose a Retentive Spatial–Spectral Transformer (RSST) for HSI denoising, which consists of the Retentive SpAtial Transformer (RSAT) block and the Retentive SpEctral Transformer (RSET) block. To enhance the adaptability of spatial–spectral representation, RSAT and RSET blocks integrate the spatial and spectral priors into the self-attention mechanism, which are formulated as a spatial decay matrix based on two-dimensional Manhattan distance and a spectral decay matrix based on one-dimensional bidirectional distance, respectively. To further improve the representation of 3D local spatial–spectral features, an Irregular Separable 3D Convolution (IrS3DC) module is integrated at the beginning of both the RSAT and RSET blocks. Additionally, RSST is configured as an asymmetric U-Net, in which the encoder and decoder blocks are implemented through the RSAT and RSET blocks, respectively. This asymmetric architecture can decouple spatial–spectral features, yielding high flexibility and low computational cost. Extensive experiments on various HSI datasets demonstrate that RSST outperforms state-of-the-art methods.