Multi-Agent reinforcement learning framework for addressing Demand-Supply imbalance of Shared Autonomous Electric Vehicle

Abstract

A critical issue in the operation of one-way station-based Shared Autonomous Electric Vehicles (SAEVs) is addressing the supply–demand imbalance. Supply-side relocations can transfer vehicles from areas with excess supply to areas with higher demand, thereby satisfying more passenger needs and increasing operator profits. To tackle the limitations of current algorithms, which fail to effectively capture similar relocation actions through spatio-temporal relationships, this paper designs a zone-based Dynamic Clustering-Driven Multi-Agent Reinforcement Learning (DC-MARL) model. The approach uses dynamic clustering to pre-cluster historical states for each time step and classifies them in real-time during training and testing. A heterogeneous action space is designed, and an optimization method is employed to determine the specific vehicles for final relocation, mapping the actions to vehicle relocation. An Entity-Agent Reshaped algorithm based on Multi-Agent Deep Deterministic Policy Gradient (EAR-MADDPG) is proposed, along with treatments to enhance cooperation among agents. Experimental results on the Suzhou Industrial Park (SIP) network demonstrate that the proposed method achieves better performance with fewer relocations compared to rule-based relocation and RL-based methods. The proposed method increases profit by 11.80% over the threshold method and by 4.25% over the advanced static clustering method.