Dynamic denoising track: Towards end-to-end multiple object tracking against attention trivialization

Abstract

Transformer-based architectures have emerged as a powerful paradigm for end-to-end multi-object tracking (MOT). However, existing frameworks typically employ static architectures with a fixed number of decoding layers, failing to adapt feature interaction levels to the varying complexities of tracking scenarios. This rigidity introduces a critical issue, which we define as attention trivialization, where excessive global interaction in deeper Transformer layers accumulates redundant noise and diminishes the discriminative power of attention. To mitigate this, we propose the Dynamic Denoising Tracking framework (DDTrack), which adaptively regulates interaction depth based on target-specific requirements. DDTrack incorporates two core components: Spatial-Temporal Early-Exit (${\text{STE}}^2$) module and Self-Guided Denoising (SGD) module. The ${\text{STE}}^2$ module mitigates noise propagation by terminating feature interactions early in the decoding process. It leverages spatio-temporal clues to perceive the complexity of foreground-background queries, enabling low-complexity samples to exit early and reduce the propagation of redundant information, thereby mitigating the attention trivialization of simple samples. The SGD module is designed for high-complexity targets. It utilizes cross-layer guidance to rectify attention trivialization. Specifically, shallow-layer queries serve as structural anchors to help deeper queries filter irrelevant noise and preserve target-specific information, thereby obtaining refined embeddings and highly discriminative attention. Finally, we jointly optimize inference depth and tracking performance through an end-to-end learning framework. Extensive experiments are conducted on DanceTrack, MOT17, and MOT20 benchmarks, achieving state-of-the-art results on several metrics and verifying the effectiveness of our proposed method.