Graph-based joint detection and tracking with Euclidean edges for multi-object video analysis

Abstract

Human detection and tracking are crucial tasks in computer vision, involving the identification and monitoring of individuals within specific areas, with applications in robotics, surveillance, and autonomous vehicles. These tasks face challenges due to variable environments, overlapping subjects, and computational limitations. To address these, we propose a novel approach using Graph Neural Networks (GNN) for joint detection and tracking (JDT) of humans in videos. Our method converts video into a graph, where nodes represent detected individuals, and edges represent connections between nodes across different frames. Node associations are established by measuring Euclidean distances between neighboring nodes, and the closest nodes are selected to form edges. This process is iteratively applied across all pairs of frames, resulting in a comprehensive graph structure for tracking. Our GNN-based JDT model was evaluated on the MOT16, MOT17, and MOT20 datasets, achieving MOTA of 85.2, ML of 11, IDF1 of 46, and MT of 65.7 on the MOT16 dataset, MOTA of 86.7 and IDF1 of 72.7 on the MOT17 dataset, and MOTA of 73.5 and IDF1 of 71.2 on the MOT20 dataset. The results demonstrate that our model outperforms existing state-of-the-art methods in both accuracy and efficiency. Through this innovative graph-based method, we contribute a robust and scalable solution to the field of human detection and tracking.