A Robust Large-Scale Multiagent Deep Reinforcement Learning Method for Coordinated Automatic Generation Control of Integrated Energy Systems in a Performance-Based Frequency Regulation Market

Abstract

To enhance the frequency stability and lower the regulation mileage payment of a multiarea integrated energy system (IES) that supports the power Internet of Things (IoT), this paper proposes a data-driven cooperative method for automatic generation control (AGC). The method consists of adaptive fractional-order proportional-integral (FOPI) controllers and a novel efficient integration exploration multiagent twin delayed deep deterministic policy gradient (EIE-MATD3) algorithm. The FOPI controllers are designed for each area based on the performance-based frequency regulation market mechanism. The EIE-MATD3 algorithm is used to tune the coefficients of the FOPI controllers in real time using centralized training and decentralized execution. The algorithm incorporates imitation learning and efficient integration exploration to obtain a more robust coordinated control strategy. An experiment on the four-area China Southern Grid (CSG) real-time digital system shows that the proposed method can improve the control performance and reduce the regulation mileage payment of each area in the IES.