SiamBSI: Hyperspectral video tracker based on band correlation grouping and spatial–spectral information interaction

Abstract

This paper proposes an innovative hyperspectral video tracking method with band correlation grouping and spatial–spectral information interaction, termed SiamBSI. The purpose is to address the insufficient information interaction between the template and search branches in Siamese network-based tracking, as well as the limited discriminative capability of existing hyperspectral image dimensionality reduction methods. The method consists of three components. First, a novel dimensionality reduction method is introduced, which groups and reduces hyperspectral images based on band correlation and spectral deviation, while generating a multi-band spectral prior mask during the reduction process. Second, to enhance the information interaction between the template and search branches of the Siamese network, an additional information interaction branch is designed. Unlike conventional approaches, this branch first performs feature correlation between the local target region in the template image and the search region, and then extracts features from the correlated results. This branch incorporates a spatial–spectral information interaction (SI) module, which integrates spatial structural features (such as edge and texture features) with deep features, effectively addressing challenges such as background clutter and scale variations. At last, a spectral prior-guided sample discrimination mechanism is designed by combining the spectral prior multi-bands mask with groundtruth, optimizing the determination of positive and negative samples, and enhancing the accuracy of classification and regression. The key advantage of the proposed tracking framework lies in overcoming the limitation of insufficient information interaction in the Siamese network. Additionally, the proposed dimensionality reduction method enhances the contrast between the target and the background while generating a spectral prior mask for subsequent tracking tasks. Experimental results validate the effectiveness of SiamBSI, achieving AUC and DP@20 of 0.663 and 0.923, respectively, outperforming ten state-of-the-art (SOTA) trackers. The conclusion indicates that the proposed method demonstrates stronger tracking capability, especially in challenging scenarios such as background clutter, with an AUC of 0.728 and a DP@20 of 0.933.