Learning Shallow-Robust Vision Transformers With Selective Distillation for Real-Time UAV Tracking

Abstract

To deploy high-performance vision transformer on resource-constrained uncrewed aerial vehicles, existing methods often adopt dynamic layer activation or layer selection mechanisms to reduce the depth of ViT and accelerate model inference. However, these mechanisms disrupt the inherent sequential feature modeling path, which can limit the modeling capacity of ViT. To alleviate this issue, we propose SRTrack, a simple yet robust tracker built upon a novel teacher-student framework. For the teacher model, we introduce a random exit training strategy that randomly terminates forward process at different ViT layers, encouraging shallow layers to undertake more modeling tasks and enhancing their discriminative capacity while preserving the sequential modeling path. For the student model, we design a selective distillation strategy to achieve effective compression. Instead of blindly relying on the conventional last-layer features, this strategy adaptively selects the most discriminative features from the enhanced teacher model as the distillation target, enabling more robust feature modeling. By combining these two strategies, SRTrack exhibits strong modeling capability even with only a few ViT layers. Extensive experiments on multiple UAV benchmarks demonstrate that SRTrack achieves a superior trade-off between tracking precision and efficiency. Our code is available at https://github.com/GXNU-ZhongLab/SRTrack