Dual Embedding Transformer Network for Hyperspectral Unmixing

Abstract

Hyperspectral unmixing is an essential task for achieving accurate perception of hyperspectral remote sensing information, aiming to overcome the limitation of spatial resolution and interpret the distribution of land features. To achieve the spatial and spectral feature representation of hyperspectral images, we propose a dual embedding transformer network (DET-Net) based on an encoder-decoder architecture, which utilizes two transformer modules, including three-view spatial attention (TVA) module with 2-D embedding and multiscale spectral band group feature fusion (BGF) module with 3-D embedding to accomplish the task of hyperspectral unmixing. In TVA module, based on 2-D embedding, we introduce a three-view attention mechanism to extract more comprehensive spatial features. In BGF module, the transformer embedding is extended to band group spatial-spectral 3-D cubed embedding and establishes a series of spectral band groups. A cross-feature fusion mechanism is adopted to achieve multiscale spatial-spectral feature decoupling. With the collaboration of these two embeddings, DET-Net effectively captures complex spatial and spectral dependencies to decouple the tridimensional unmixing feature representation. Experimental results on synthetic and real datasets demonstrates the generalization performance of the proposed method, and the ablation experiments confirm the effectiveness of the TVA and BGF modules.