Game-Guided Multi-Agent Reinforcement Learning Approach for Vehicle Decision-Making in Narrow-Road Meeting With Hybrid Interaction

Abstract

Narrow-road meeting is one of the most challenging interactive driving scenarios, yet it has not been adequately studied in previous research. Current multi-agent reinforcement learning (MARL)-based approaches face critical limitations when addressing head-on conflict scenarios. One major limitation is the assumption of single-modal interactions, such as cooperative behavior patterns, which fail to depict the complexities of negotiation dynamics. Additionally, the convergence of MARL models is not guaranteed, often leading to local optima or deadlocks in extreme conflict. To address these challenges, this work proposed a game-guided MARL framework that integrates the Stackelberg game theory with a Multi-agent Proximal Policy Optimization (MAPPO) model. By incorporating leader-follower roles into the actor network, the model can handle hybrid interaction modalities. The role of an agent is evaluated based on their priorities regarding right-of-way, reversing conditions, and urgency level. Based on this, the agents make decisions by role identification in a ‘think twice, act once’ way. For training, we created scenarios in which the ego vehicle interacts with random oncoming traffic flows. For each background social vehicle, Social Value Orientation (SVO) is introduced alongside the intelligent driver model (IDM) for controlling. Building on the trained MARL model, a self-play adversarial learning mechanism is further designed to enhance the strategies’ robustness to mixed traffic. Experimental results demonstrate that the proposed game-guided MAPPO model achieves the best performance in terms of reward and success rate. After adversarial training, the model gains greater robustness to varied social behaviors.