An improved deep learning architecture for multi-object tracking systems

Robust and reliable 3D multi-object tracking (MOT) is essential for autonomous driving in crowded urban road scenes. In those scenarios, accurate data association between tracked objects and incoming new detections is crucial. This paper presents a tracking system based on the Kalman filter that uses a deep learning approach to the association problem. The proposed architecture consists of three neural networks. First, a convolutional LSTM network extracts spatiotemporal features from a sequence of detections of the same track. Then, a Siamese network calculates the degree of similarity between all tracks and the new detections found at each new frame. Finally, a recurrent LSTM network is used to extract 3D and bounding box information. This model follows the tracking-by-detection paradigm and has been trained with track sequences to be able to handle missed observations and to reduce identity switches. A validation test was carried out on the Argoverse dataset to validate the performance of the proposed system. The developed deep learning approach could improve current multi-object tracking systems based on classic algorithms like the Kalman filter.