3D Car Tracking using Fused Data in Traffic Scenes for Autonomous Vehicle

Car tracking in a traffic environment is a crucial task for the autonomous vehicle. Through tracking, a self-driving car is capable of predicting each car’s motion and trajectory in the traffic scene, which is one of the key components for traffic scene understanding. Currently, 2D vision-based object tracking is still the most popular method, however, multiple sensory data (e.g. cameras, Lidar, Radar) can provide more information (geometric and color features) about surroundings and show significant advantages for tracking. We present a 3D car tracking method that combines more data from different sensors (cameras, Lidar, GPS/IMU) to track static and dynamic cars in a 3D bounding box. Fed by the images and 3D point cloud, a 3D car detector and the spatial transform module are firstly applied to estimate current location, dimensions, and orientation of each surrounding car in each frame in the 3D world coordinate system, followed by a 3D Kalman filter to predict the location, dimensions, orientation and velocity for each corresponding car in the next time. The predictions from Kalman filtering are used for re-identifying previously detected cars in the next frame using the Hungarian algorithm. We conduct experiments on the KITTI benchmark to evaluate tracking performance and the effectiveness of our method.