DiffHSR: Unleashing Diffusion Priors in Hyperspectral Image Super-Resolution

Abstract

Hyperspectral images provide rich spectral information and have been widely applied in numerous computer vision tasks. However, their low spatial resolution often limits their use in applications such as image segmentation and recognition. In previous works, generating high-resolution hyperspectral (HR-HS) images required the use of low-resolution hyperspectral (LR-HS) images and high-resolution RGB (HR-RGB) images as priors, which increases the cost of data collection and may lead to measurement and calibration errors in practical applications. Although the currently popular CNN-based single hyperspectral image super-resolution (single HS-SR) methods have improved performance, they are not flexible enough to process images with different degradation. From a visual perspective, the generated super-resolution images exhibit a significant smudging effect due to the loss of information. Leveraging multi-modal techniques and generative prior, we propose DiffHSR that marks a significant leap in LR-HS images super-restoration without HR-RGB. Additionally, we have established a connection between hyperspectral images and the RGB image-based generative model tasks using low-cost data and fine-tuning approaches, which creates a novel paradigm. Comprehensive experiments have demonstrated that our proposed method achieves strong visual performance and competitive results in term of quantitative metrics and perceptive quality.