Dual-Branch Cross Weighting Network for Multimodal Hyperspectral Unmixing

Abstract

Hyperspectral unmixing has recently attracted much attention in the field of spectral image analysis. Unsupervised methods based on autoencoder can achieve excellent unmixing performance. However, these methods have unsatisfactory unmixing results for different substances with similar materials in complex scenarios. In this letter, we propose a new dual-branch cross weighting network (DCWNet) for multimodal hyperspectral unmixing. It can not only use the spectral feature information of spectral images but also acquire the spatial feature information of light detection and ranging (LiDAR) data simultaneously. Specifically, we build a spatial channel augmentation (SCA) block to help the network acquire spatial information more accurately from the horizontal, vertical, and channel aspects, respectively. We further construct an adaptive feature selection module for effectively using the spectral feature information and spatial feature information to better focus on the discrimination of materials in the scene through weighting fusion. Experimental results on two real multimodal datasets demonstrate the competitiveness and effectiveness of our proposed DCWNet in comparison to five representative methods.