Multi-agent modeling for energy storage charging station scheduling strategies in the electricity market: A cooperative learning approach

Abstract

With integration of an energy storage system (ESS), an energy storage charging station serves as pivotal intermediaries between the smart grid and electric vehicles (EVs). This station utilizes the ESS to enhance grid stability and facilitate energy management. Participation in electricity market transactions offers revenue opportunities for charging stations, but it also introduces operational challenges, due to fluctuating electricity market prices and diverse energy demands and supplies. In this paper, we study the operation strategy optimization problem for the charging station, addressing economic and service challenges influenced by market volatility and energy diversity. The optimization objective considers not only maximizing economic benefits from the electricity market and EV services but also minimizing penalties associated with EV service quality. We propose a model that accounts for the dynamics of the electricity market, uncertainties from EV demands, and disturbances from green power generation, optimizing the power scheduling of the ESS and multiple charging piles (CPs) to determine transaction power in the market. The cooperative scheduling strategies for the ESS and CPs are learned using the proposed heterogeneous Multi-agent Deep Deterministic Policy Gradient method. This approach features distributed agents learning to determine decision variables for both the ESS and CPs, while a joint critic network assesses the station’s overall objectives to guide their cooperative learning. The proposed method was tested against three state-of-the-art benchmark methods, which showed our method achieves better results.