A channel- adaptive and plug-and- play framework for hyperspectral image analysis

Abstract

HyperSpectral Image (HSI) reflects rich properties of matter and facilitates distinguishing various objects, demonstrating substantial potential in a wide range of applications, including medical diagnosis and remote sensing. However, HSI exhibits variable number of channels due to the variations in acquisition equipments, which makes existing HSI analytical methods fail to utilize data from multiple equipments. To address this challenge, we first distill HSIs with varying channels into principal and residual components. We then develop a Fusion-Guided Network (FGNet) to transform the two distilled components into fused images with a fixed number of channels and perform channel-adaptive HSI analysis. To enable the fused images to maintain intensity, structure, and texture information in the original HSI, we generate pseudo labels to supervise the fusion. To facilitate the FGNet to extract more representative features, we further design a low-rank attention module (LGAM), leveraging the low-rank prior of HSI that few key information can represent a large amount of data. Moreover, the proposed framework can be applied as a plug-in to existing HSI analysis methods. We conducted extensive experiments on five HSI datasets including medical HSI segmentation task and remote sensing HSI classification task, which demonstrates the proposed method outperforms the state-of-the-art methods. We further experimentally identified that existing works can be seamlessly incorporated with our framework to achieve channel-adaptive ability and boost analytical performance. Code is available at https://github.com/hnsytq/FGNet.