ISSLDIA: Internal structure self-learning and low-dimensional image assistance for single hyperspectral image super-resolution

Abstract

The single hyperspectral image super-resolution (HSI-SR) plays an important role in enhancing the spatial resolution of HSIs by only using their corresponding low-resolution hyperspectral images (LR-HSI). Most single HSI-SR methods have achieved great success recently and can be simply divided into learning-based and model-based methods. However, because of the difficulty of obtaining sufficient and identically distributed trainable HSIs, the former usually cut a portion of LR-HSIs and assumes that the corresponding high-resolution hyperspectral images (HR-HSI) are known to learn a learning module, which is difficult to achieve in reality. Although the latter do not require the training phase, they will inevitably influence the reconstruction accuracy because of the absence of a training phase. In this paper, we propose an internal structure self-learning low-dimensional image assistance (ISSLDIA) HSI-SR method by fully capturing the internal structure of LR-HSIs and similar low-dimensional images to reconstruct HR-HSIs. First, because the LR-HSI and HR-HSI contain complete spectral information, that is, the spectral correlation information of the LR-HSI and HR-HSI are similar to each other, we can extract the spectral information of the LR-HSI to guide the spectral reconstruction of the HR-HSI. Second, although the trainable HR-HSIs are hard to require, the high-resolution low-dimensional images are relatively easy to require. Both of them have similar spatial structures in small patches. We can learn the spatial structures from these low-dimensional images to assist in the spatial structure reconstruction of HR-HSI. Third, the LR-HSI is the downsampled version of the HR-HSI. Similarly, we can also get the downsampled version of LR-HSI by using the same downsampled operator. The similarity relationship between the LR-HSI and the downsampled LR-HSI is similar to that between the HR-HSI and the LR-HSI. Hence, we used the learned similarity relationship to the reconstruction of the HR-HSI. Finally, we integrate the three modules to generate an HSI-SR module, which is optimized using the alternating direction method of multipliers (ADMM). The experimental results on four HSI datasets show the superiority of the proposed method to some other traditional and advanced SR methods. The demo code can be found at https://github.com/jinhaoRen/ISSLDIA.