Autonomous On-ramp Merge Strategy Using Deep Reinforcement Learning in Uncertain Highway Environment

Abstract

On-ramp merge is a complex traffic scenario in autonomous driving. Because of the uncertainty of the driving environment, most rule-based models cannot solve such a problem. In this study, we design a Deep Reinforcement Learning (DRL) method to solve the issue of ramp merges in uncertain scenarios and modify the structure of the Twin Delayed Deep Deterministic policy gradient algorithm (TD3), using Long Short-Term Memory (LSTM) to select an action based on temporal information. The proposed method is applied in the on-ramp merge and verified in the Simulation of Urban Mobility (SUMO). Results show that the proposed method performs significantly better generalization in uncertain traffic scenarios.