SASU-Net: Hyperspectral video tracker based on spectral adaptive aggregation weighting and scale updating

In order to address the challenge of scale variation in hyperspectral video tracking, we propose a novel tracker based on spectral adaptive aggregation weighting and scale updating. First, band adaptive aggregation weighting is used to reduce the dimensionality of the hyperspectral image and generate a spectral prior mask. The dimensionality-reduced image is then fed into the ResNet50 network for feature extraction. The features of the target and search regions are respectively directed to the encoder and decoder. Simultaneously, the spectral prior mask is input into the decoder for prior correction. The output from the decoder, fused vector, is subjected to anchor-free prediction for precisely determining the target position. SASU-Net incorporates a scale aware update module and a segmented template update strategy, which is founded on scale evaluation. At last, the target classification score and scale score are evaluated to determine whether to update the template. Through the novelty utilization of spectral adaptive aggregation, decoding based on prior masks, and the fusion with scale sensing and update strategy, SASU-Net shows a significant advantage in tackling multiple challenges, especially scale variation. Experimental results illustrate the superior performance of the proposed framework in comparison with the benchmark hyperspectral video trackers specially in effectively addressing scale variation challenges. The source code along with model weights of this work is publicly available at https://github.com/CodeMANz11/SASUNet.