Research on vertical strategy for left turn at signal-free T-shaped intersections based on multi-layer reinforcement learning methods

Abstract

The task of executing left turns at signal-free T-shaped intersections without protective signals poses a critical challenge in the realm of autonomous driving. Conventional rule-based approaches tend to be excessively cautious, rendering them inadequate for effectively managing driving tasks within unpredictable T-shaped intersection environments. In the case of complex traffic scenarios, a single model is less effective in convergence and has a lower pass rate and poorer safety. Thus, this study introduces a multi-layer reinforcement learning model, employing D3QN (Dueling Double DQN) and TD3 (Twin Delayed Deep Deterministic policy gradient algorithm) for advanced behavioral decision-making and vertical acceleration planning, respectively. In our experimental investigation, we designed four simulation scenarios based on the driving behavior of the Carla simulator to replicate real-world driving conditions. Verification and test simulation outcomes substantiate that, in comparison to other single-trained reinforcement learning models, the multi-layer reinforcement learning model proposed in this study attains the highest success rate. Specifically, the pass rate in the verification scenario, consistent with the training conditions, achieves an impressive 99.5%. Furthermore, the pass rate in the comprehensive test scenario reaches 89.6%. These experiments unequivocally demonstrate the considerable enhancement in T-shaped intersections pass rates achieved by the proposed method while ensuring both traffic efficiency and safety.