Snapshot hyperspectral imaging with high spatial resolution based on transformers.

Abstract

Snapshot spectral imaging based on spectral response encoding has become a research focus due to its miniaturization and simple optical layout. Nevertheless, the limited pixel density of image sensors has become a technical bottleneck restricting spectral and spatial resolution. To address this problem, efficient spectral encoding methods and high-precision spectral reconstruction algorithms are crucial. In this study, we propose the theory of spectral information entropy transfer and find that the encoding loss of the target spectral cube's information entropy is the main factor restricting the spatial and spectral resolution of the reconstructed spectral cube. Based on this theory, we have developed the hyperspectral imaging transformers network (HSITNet), based on transformers. Compared to other models, HSITNet has a broader field of vision, effectively reducing the joint distribution information entropy of the spectral cube, and achieves higher spectral reconstruction quality. To fully explore the encoding performance, we introduce an algorithm to jointly optimize the encoding and decoding strategies. This mutual adaptation enhances the imaging quality of the spectral cube and enables the automatic design of encoding devices. In data experiments, we successfully reconstructed data cubes with 151 spectral channels, achieving pixel-level spatial resolution without mosaic. Reconstruction results show superior performance with metrics: MSE = 1.21 × 10^-4, SAM = 0.041, PSNR = 39.72, and SSIM = 0.95, thereby realizing snapshot hyperspectral imaging with no spatial resolution degradation.