A Privacy-Preserving Federated Reinforcement Learning Method for Multiple Virtual Power Plants Scheduling

Abstract

The application of federated learning in Virtual Power Plants (VPPs) addresses the data silo issue between VPPs and enhances their ability to cope with nonlinear and stochastic scheduling characteristics, which enables VPPs better accommodate distributed energy resources and flexible loads while participating in frequency regulation services. However, although existing federated learning methods strive to solve privacy protection issues, the plaintext transmission of gradients still exposes sensitive data to the threat of curious power control centers and external inference attacks. Therefore, a privacy-protected horizontal federated reinforcement learning approach for multi-VPP optimal scheduling is proposed in this paper. Firstly, a cost-based global optimization scheduling model for multiple VPPs is constructed, modeling the internal scheduling process of VPPs as a Markov decision process. Then, an improved secure horizontal federated multi-VPP collaborative training method is presented, and local models are trained using the Deep Transformer Q-Network algorithm, with local differential privacy and CKKS homomorphic encryption implemented to ensure privacy protection. Finally, a case study is conducted using frequency regulation ancillary service market data and the IEEE-39 bus system structure. Simulation results show that the proposed approach outperforms similar algorithms, achieving high levels of privacy protection and economic operation for VPPs.