WDTM-CL: Efficient Wavelet-based dual Transformer model with contrastive learning for spectral reconstruction from RGB images

Abstract

Spectral reconstruction from RGB images presents a significant challenge due to its ill-posedness. Existing Transformer-based methods for this task are usually computationally inefficient, as their complexity increases quadratically with the spatial resolution of the input. Furthermore, existing methods focused more on preserving local image structures, while their global context and non-local correlations are not exploited explicitly, resulting in degraded hyperspectral image (HSI) reconstruction. To tackle these issues, we propose an efficient Wavelet-based dual Transformer model (WDTM) with contrastive learning (CL) dubbed WDTM-CL for spectral reconstruction in this paper. Our WDTM-CL incorporates a dual attention mechanism adept at capturing both the non-local spatial similarities and the global spectral correlations within HSI. Wavelets are used for signal decomposition, to preserve the essential details of the feature maps, enabling effective multi-head self-attention learning and improving computational efficiency. Finally, we employ a patch-wise contrastive loss for hyperspectral data, to ensure structural fidelity in the reconstructed HSI by promoting patch-wise consistency with the ground truth HSI. This strategy captures the spectral and spatial information more accurately. Extensive experimental validation across a range of benchmark datasets shows that our proposed WDTM-CL achieves state-of-the-art performance in spectral reconstruction tasks.