A new multi-object tracking algorithm based on Sparse Detection Transformer

Abstract

Multi-object tracking (MOT) is crucial for intelligent surveillance and autonomous driving. However, existing Transformer-based methods often suffer from an accuracy-efficiency trade-off due to high computational complexity, limiting real-time applicability. To address this, we propose SparseDeTrack (Sparse Detection Tracking), an efficient MOT framework based on the tracking-by-detection (TBD) paradigm. In detection, we employ a sparse token Transformer with a 30 % token retention rate, effectively reducing computational cost while retaining essential features. In tracking, we remove the Re-Identification (ReID) module and enhance the Extended Kalman Filter (EKF) by directly predicting the width and height instead of the aspect ratio of bounding boxes, improving both localization accuracy and nonlinear motion modeling. Furthermore, ByteTrack (Multi-Object Tracking by Associating Every Detection Box) is integrated for secondary association, increasing robustness under occlusion. We conduct extensive experiments on MOTChallenge 17 (MOT17), MOTChallenge 20 (MOT20), and DanceTrack benchmarks. On the MOT17 test set, SparseDeTrack achieves a Multiple Object Tracking Accuracy (MOTA) of 75.4, outperforming Transformer-based methods such as MOTR (End-to-End Multiple-Object Tracking with Transformer), Trackformer (Multi-Object Tracking with Transformers), and TransTrack (Multiple Object Tracking with Transformer) by 2.0, 1.3, and 0.2 points, respectively, while attaining a high inference speed of 44.5 frames per second (FPS), balancing accuracy and efficiency. It reaches 65.6 MOTA on crowded MOT20 and 89.1 MOTA on nonlinear-motion DanceTrack, comparable to state-of-the-art methods. These results confirm that SparseDeTrack delivers both high-precision tracking and real-time inference in complex scenarios, making it a promising solution for real-world applications in intelligent surveillance and autonomous driving.