Real-time object detection and 3D scene perception in self-driving cars

Reliable autonomous urban driving hinges upon the vehicle's ability to perceive and navigate the environment. This research paper emphasizes designing and implementing a vision-based perception system for NUSTAG self-driving car. The primary task is the implementation of 3D bounding box estimation and depth perception using a stereo camera feed to estimate the positions of cars, bikes, and pedestrians. Moreover, road signs and traffic lights are detected using 2D object detection and classification. The major challenge to implement all these deep learning algorithms in parallel in the NVIDIA Jetson Xavier development kit is achieved by optimizing the models to perform inference in real-time. This is accomplished using the TensorRT framework employing ROS interface. The models have been trained for our requirements to yield efficient results within our operational design domain.