The Design of a Multi-Task Learning System for Autonomous Vehicles

Abstract

Recently, a lot of research and applications regarding autonomous vehicles have been invested in and developed. These applications have many complex scenarios to handle such as lane segmentation, object detection, traffic sign recognition, and steering control prediction. Many methods handle these tasks separately. Despite the excellent performance these methods achieve, processing these tasks one after another takes a longer time than tackling them all at once. So, in this paper, to reduce the inference time of the autonomous driving system, we proposed a multi-task framework to conduct three tasks: lane segmentation, object detection, and traffic sign recognition simultaneously. Our framework is composed of one encoder for feature extraction and two decoders to handle specific tasks. We only use one encoder for multiple tasks because these tasks complement each other, we hope that the information can be shared among these tasks through the single encoder to improve the performance of each task and also to reduce the amount of data required for training. The decoders include a detection decoder and a segmentation decoder. The detection decoder is designed to detect objects and recognize traffic signs. On the other hand, the segmentation decoder is designed to focus solely on the task of separating the drivable area. By testing on the challenging Carla dataset, our model shows that it can achieve better results compared to state-of-the-art methods. Besides, experimental results also show that, compared with solving tasks independently, our framework can achieve similar performance but greatly reduce processing time.