DTTrack: Metric-Guided Dynamic Template Tracking Network

Abstract

While unified weight-sharing architectures have shown strong versatility across RGB-T, RGB-D, and RGB-E tracking, accommodating heterogeneous data often weakens fine-grained target discriminability and inevitably leads to feature dilution in long-term tracking. Conventional update strategies also struggle to maintain reliable memory under frequent shifts in modality dominance and heterogeneous noise, thereby limiting tracking performance. To address these issues, this paper proposes the Metric-Guided Dynamic Template Tracking Network (DTTrack). By constructing an independent, modality-agnostic metric space, DTTrack enables effective long-term memory modeling in a single-stream framework without modifying the backbone. Specifically, a Metric-Guided Synergistic Decision Head (MSDH) is introduced to decouple localization and discriminative features explicitly. Additionally, a Sequence-Aware Adaptive Sampling (SeAS) strategy is developed to encourage identity-invariant learning across temporal scales. Furthermore, a quality- and diversity-aware dynamic update mechanism is designed in the learned metric space to select high-confidence templates and suppress heterogeneous noise. Extensive experiments on five benchmark datasets across RGB-T, RGB-D, and RGB-E tracking demonstrate that DTTrack achieves state-of-the-art or highly competitive performance.