Decision-Making Strategy Using Multi-Agent Reinforcement Learning for Platoon Formation in Agreement-Seeking Cooperation

Abstract

Among the four classes of cooperative driving automation defined in [1], agreement-seeking cooperation appears to be a promising option for achieving higher cooperation levels with general passenger vehicles. Because agreement-seeking cooperation allows connected and automated vehicles (CAVs) to decide whether or not to participate in cooperative driving, it is necessary for CAVs to have intelligent decision-making strategies. This work develops a farsighted, interaction-aware decision-making strategy using multi-agent reinforcement learning (MARL). A MARL system is formulated with unique state and action spaces reflecting agreement-seeking interactions. A state–action–reward–state–action (SARSA) algorithm is applied to learn the action-value function of each CAV. Simulation results show that using a MARL-based decision-making strategy increases agreement rates by 52% on average and cooperation time by 50%. The higher cooperation rates lead to higher energy efficiency: 5.5% more energy saving than heuristic decision-making.