Safe control strategy for energy storage cluster assisted load frequency control based on reinforcement learning

Abstract

The large-scale integration of renewable energy into the power grid introduces strong stochastic disturbances, posing new challenges to the safety of load frequency control (LFC). To deal with this issue, a safety control strategy is proposed for lithium-ion energy storage cluster into LFC. First, to achieve efficient frequency control with the energy storage cluster, a command allocation strategy for energy storage cluster and a control strategy for units are proposed, with comprehensive consideration of the state of charge, state of health and the real-time grid frequency deviation. Next, both the maximum frequency deviation (MFD) and the rate of change of frequency (RoCoF) are picked as dynamic response performance indexes to ensure frequency safety. Then, a novel LFC controller based on Safety Enhanced Deep Deterministic Policy Gradient (SE-DDPG) reinforcement learning algorithm is designed. The safety model of SE-DDPG which integrated with safety prediction network and intrinsic curiosity module (ICM) can enhance the exploratory capability while improving the safety and reliability of the policy. Finally, the effectiveness of the proposed safe LFC strategy is validated by numerical simulation. Compare with conventional proportional integral control, the proposed strategy reduces the MFD and the root mean square frequency deviation by 41.38 % and 22.74 % in the random noise scene. In the step load scene, MFD and the max RoCoF are reduced by 46.88 % and 48.15 %.