SiamSCT: Spatial-Channel Saliency and Temporal Fusion Network for Real-Time Aerial Tracking

Abstract

Visual tracking on uncrewed aerial vehicle (UAV) platforms is a fundamental and crucial visual task. Compared to conventional tracking tasks, aerial tracking faces specific challenging scenarios due to its unique perspective. Although existing aerial trackers have demonstrated promising performance, they remain limited in capturing spatial-channel saliency across branches and effectively utilizing historical information. To address these issues, this letter proposes a spatial-channel saliency and temporal fusion network (SiamSCT), which aims to enhance feature representation for efficient and accurate aerial tracking. Specifically, SiamSCT introduces a weight-shared spatial saliency block (SSB) to strengthen the spatial feature representation across the tracking network’s branches. In addition, a light channel aware module (CAM) is designed to facilitate deep feature interaction across branches at the channel level, further enhancing feature discriminability. Finally, using a historical similarity response fusion strategy, SiamSCT achieves more stable and reliable tracking responses, effectively tackling complex aerial scenarios. Extensive experiments on several authoritative aerial tracking datasets demonstrate that SiamSCT outperforms state-of-the-art (SOTA) trackers. Furthermore, SiamSCT achieves a tracking speed of 133 frames/s on NVIDIA RTX 3060Ti, proving its excellent performance in real time.