Research on Platoon Agent Controller Based on Twin Delayed Deep Deterministic Policy Gradient Algorithm*

At present, in the research of platoon controller, most of it is based on Model Predictive Control, PID and other control algorithms under the condition of considering communication delay, topology structure and string stability. With the development of Deep Reinforcement Learning, agents can decide the controlled variable according to the state, which is very beneficial to the control of complex systems. Therefore, Twin Delayed Deep Deterministic policy gradient algorithm is used to control the agent platoon, and regards the information interaction within the platoon as a decision process with Markov properties. Matlab and Sumo are used to build a training platform, so as to train the function approximator that maps the state quantity to the action. In the design of the reward function, the state quantity involved in the string stability is given different weights to achieve the string stability from various aspects. Compared with the linear dynamic model, we use the 5-Degree of Freedom nonlinear dynamic model to make the dynamic characteristics of the vehicle more real. The simulation results show that the platoon agent trained by the Twin Delayed Deep Deterministic policy gradient algorithm can guarantee a certain string stability.