Unleashing Correlation and Continuity for Hyperspectral Reconstruction From RGB Images.

Abstract

Reconstructing hyperspectral images (HSIs) from RGB images can yield high spatial resolution HSIs at a lower cost, demonstrating significant application potential. This article reveals that local correlation and global continuity of the spectral characteristics are crucial for HSI reconstruction tasks. Therefore, we fully explore these interspectral relationships and propose a correlation and continuity network (CCNet) for HSI reconstruction from RGB images. To model the correlation of local spectra, we introduce the groupwise spectral correlation modeling (GrSCM) module, which efficiently establishes spectral band similarity within a localized range. To model the continuity of the global spectrum, we design the neighborhood-wise spectral continuity modeling (NeSCM) module, which employs memory units to recursively model the progressive variation characteristics at the global level. In order to explore the inherent complementarity of these two modules, we design the patchwise adaptive fusion (PAF) module to efficiently integrate global continuity features into the spectral features in a patchwise adaptive manner. These innovations improve the quality of the reconstructed HSI. We perform comprehensive comparative and ablation experiments on the NTIRE2020, NTIRE2022, and CAVE datasets for the spectral reconstruction task. Compared with current advanced spectral reconstruction algorithms, the proposed method achieves state-of-the-art (SOTA) performance.