A velocity adaptive steering control strategy of autonomous vehicle based on double deep Q-learning network with varied agents

Abstract

Autonomous vehicle steering control is sensitive to the vehicle driving speed and traditional model-based approaches are limited by the accuracy of the control model in various driving speed scenarios. To address these challenges, this study proposes a model-free control strategy based on deep reinforcement learning (DRL). In this strategy, the improved double deep Q-learning network (DDQN) with varied agents is employed for steering control to minimize the tracking errors across varying speeds. According to the kinematic characteristics of the vehicle, a dynamic action space is applied to enhance the tracking capability at high speeds. Furthermore, to ensure the output of the agent is more stable, a velocity adaptive reward function is designed by incorporating an action penalty factor. The performance of the proposed strategy is evaluated through simulation and experimental comparisons with other existing algorithms at a double-lane change maneuver. The results demonstrate that the DDQN-based strategy can effectively adapt to various vehicle speeds and perform the tracking task more accurately and stably. Finally, the feasibility of this strategy is verified using an actual prototype vehicle.