Deep RGB-guided generative network for unsupervised hyperspectral image super-resolution

Hyperspectral image (HSI) super-resolution (SR) aims to mathematically generate a high spatial resolution hyperspectral (HR-HS) image by merging the degraded observations: a low spatial resolution hyperspectral (LR-HS) image and a high spatial resolution multispectral or RGB (HR-MS/RGB) image. Currently, deep convolution network-based paradigms have been extensively explored to automatically learn the inherent priors of the latent HR-HS images and have shown remarkable performance progress. However, existing methods usually are realized in a fully supervised manner and have to previously prepare a large external dataset containing the degraded observations: the LR-HS/HR-RGB image and its corresponding HR-HS ground truth, which are difficult to collect, especially in the HSI SR scenario. To this end, this study proposes a novel unsupervised HSI SR method by using only the observed degradation data without any other external sample. Specifically, we use a deep RGB-guided generative network to generate the target HR-HS image with an encoder-decoder-based network. Since the observed HR-RGB image has a more detailed spatial structure, which may have better compatibility with the 2D convolution operation, we take the observed HR-RGB image as the network input to serve as the conditional guidance, while using the degraded observations to construct the loss function to guide the network learning. Experimental results on several benchmark HS image datasets demonstrate that the proposed unsupervised method achieves superior performance over various SoTA paradigms.