Federated Multi-Agent Deep Reinforcement Learning-Based Competitive Pricing Strategy for Charging Station Operators

Abstract

With the rapid advancements in transportation electrification, the proliferation of electric vehicles (EVs) has interconnected power and transportation networks, forming the vehicle-traffic-power nexus. By setting charging prices, charging station operators (CSOs) can effectively guide the charging behavior of EVs, alleviate grid stress, and enhance profitability. This paper proposes a Nash-Stackelberg-Nash (N-S-N) game model to investigate the competitive charging pricing strategy for CSOs. We establish the stochastic user equilibrium with the elastic demand traffic assignment problem (SUE-ED-TAP) model to account for users' incomplete rationality and perception errors regarding trip costs. Furthermore, to protect the privacy of both CSOs and EV users, a federated multi-agent deep reinforcement learning-based solution method is proposed to solve this problem. In this method, a non-profit aggregator is introduced to exchange neural network parameters among agents, enabling privacy-preserving and collaborative learning without sharing CSOs' data. Case studies on two test systems show that the proposed method achieves higher profits compared to existing algorithms.