CCMA: A framework for cascading cooperative multi-agent in autonomous driving merging using Large Language Models

Abstract

Traditional Reinforcement Learning (RL) suffers from challenges in replicating human-like behaviors, generalizing effectively in multi-agent scenarios, and overcoming inherent interpretability issues. These tasks become even more difficult when they require a deep understanding of the environment, coordination of agents’ intentions and driving styles across various scenarios, and the overall optimization of safety, efficiency, and comfort in dynamic environments. Recently, Large Language Model (LLM) enhanced methods have shown promise in improving generalization and interoperability. However, these approaches primarily focus on single-agent scenarios and often neglect the necessary coordination among multiple road users. Therefore, in this paper, we introduce the Cascading Cooperative Multi-agent (CCMA) framework, designed to address these challenges by enhancing human-like behaviors and fostering multi-level cooperation across diverse multi-agent driving tasks, ultimately improving both micro and macro-level performance in complex driving environments. Specifically, the CCMA framework integrates RL for individual interactions, a fine-tuned LLM for regional cooperation, a reward function for global optimization, and the Retrieval-augmented Generation mechanism to dynamically optimize decision-making across complex driving scenarios. Our experiments demonstrate that our CCMA method not only enhances human-like behaviors and interpretability, but also outperforms other state-of-the-art RL methods in multi-agent environments. These findings highlight the significant impact of cascading coordinated communication and dynamic functional alignment in advanced, human-like multi-agent autonomous driving environments. Our project page is https://miaorain.github.io/rainrun.github.io/.