Hyperspectral Image Few-Shot Classification Based on Spatial–Spectral Information Complementation and Multilatent Domain Generalization

Abstract

Hyperspectral image (HSI) few-shot classification aims to classify HSI samples of novel categories with limited training HSI samples of base categories. However, current methods suffer from two issues: first, ignoring the complementary relationship between spatial and spectral information; and second, performance degradation on base categories due to excessive focus on novel categories. This article proposes a spatial–spectral information complementation and multilatent domain generalization-based framework (SIM). Specifically, given samples of base (novel) categories, a spatial–spectral feature extraction network is designed to extract their spatial–spectral features, which includes two steps. First, multiple spatial–spectral information complementation modules (SSICs) are stacked to extract the complementary features with different scales. Note that each SSIC extracts features with spatial and spectral information, and adopts a spatial–spectral information transmission unit to cross-transmit spatial and spectral information between these two types of features, thus achieving information complementation. Second, a multiscale feature fusion module is utilized to calculate the classification influence scores of the multiscale complementary features to perform layer-by-layer feature fusion, thus obtaining spatial–spectral features. Afterward, the spatial–spectral features are fed into a classification head to obtain the classification results. During training, a multilatent domain generalization network (MLDGN) is designed, which iteratively assigns pseudodomain labels to all samples, and calculates the sample discrimination loss. SIM combines the sample discrimination loss with the classification losses for training. Thus, SIM can extract spatial–spectral features with domain invariance, alleviating the performance degradation on base categories. Extensive results on four HSI datasets demonstrate that SIM outperforms state-of-the-art methods.