MSTGT: Multi-scale spatio-temporal guidance for visual tracking

Abstract

Addressing the challenge of target tracking in complex scenarios with limited data samples is a highly significant research endeavor. Nevertheless, most trackers primarily concentrate on intricate model architectures or template updating strategies, overlooking the depth of training sample exploitation and the efficient utilization of spatio-temporal target information. To alleviate the above problem, we propose a novel visual tracking framework tailored for complex scenarios, named MSTGT, which integrates mixed data sampling with multi-scale spatio-temporal guidance. Specifically, we innovatively employ a video sequence sampling and feature mixing strategy to simulate complex scenarios, enhancing the representation of video sequences. Concurrently, our multi-scale visual cue encoder harnesses multi-scale target information to fortify feature representation and cue construction. Furthermore, our multi-scale spatio-temporal guidance encoder, a groundbreaking approach, seamlessly integrates spatial and temporal dimensions with multi-scale information, precisely guiding the prediction of target trajectories. This not only bolsters the handling of intricate motion patterns but also circumvents the need for intricate online updating strategies. MSTGT achieves SOTA performance on six benchmarks, while running at real-time speed. Code is available at https://github.com/capf-2011/MSTGT.