Channel-wise mask learning based mixing transformer for spectral compressive imaging

Abstract

Single disperser coded aperture spectral imaging (SD-CASSI) is well-known for its simple optical path that efficiently acquires spectral images. However, reconstructing hyperspectral images from their measurement scenes is an ill-posed and challenging problem. By applying deep learning methods to solve this ill-posed issue, it becomes possible to reconstruct high-quality hyperspectral images from measurement images in real time. However, mainstream models typically use an encoder–decoder structure, connecting the output of the encoder and the input of decoder only along the channels. This limits the ability of network to learn detailed image information. In addition, since the planar image sensor array causes varying wavelengths to experience different optical path differences after dispersion, the actual mask cannot be derived solely from a single known mask through different dispersion steps. To address these issues, this paper proposes a deep unfolding method called the channel-wise mask learning based mixing Transformer network (CML-MT). We design a denoising model based on window attention and a dual block, using the dual block as the decoder to fully utilize information from the encoder layers. Additionally, we introduce a channel-wise degradation mask learning module that implicitly learns to approximate the latent real mask under the constraint of multi-stage reprojection loss. Experimental results demonstrate that with these solutions, our model, extended to only three stages, is competitive with state-of-the-art models and excels in reconstructing details and textures in real-world scenarios.