Vision-Based Driving Decision Making Using Multi-Action Deep Q Network

Abstract

The performance improvement of perception algorithms and integrated validation of driving decision making remains challenging in the fields of computer vision and intelligent transportation systems. In this paper, we propose a novel vision-based framework for driving decision making, which is composed of three stages: object perception, lane line perception and driving decision making. For the object perception stage, an improved object perception model named CenterNet-ARA is developed, composing of a new adversarial training method, a receptive field enhancement module and an adaptive sample allocation equalization strategy to fuse multi-scale feature maps. For the lane line perception stage, a lane line perception method named Lite-MobileTR is proposed, which contains an improved Lite-MobileNetV3 encoder and an improved lite-transformer decoder. Moreover, a noise removal task is incorporated to alleviate the problem of slow convergence speed caused by Hungarian loss function. For the driving decision making stage, a new Multi-Action DQN is proposed utilizing a vehicle curriculum learning strategy and a curiosity exploration strategy to alleviate the problem of random exploration in the learning process. The proposed framework is evaluated on the Tusimple, CULane, TSD-max, and KITTI datasets. Finally, an integration verification is performed in Carla simulator to validate the driving decision making process. The experimental results well demonstrate the effectiveness of the proposed framework.