A Benders-Combined Safe Reinforcement Learning Framework for Risk-Averse Dispatch Considering Frequency Security Constraints

Abstract

Risk-averse dispatch considering frequency security constraints (FSC-RD) mitigates power supply-demand imbalance risks and frequency instability hazards. To effectively address the highly complex, multi-task coupled FSC-RD, this brief proposes a Benders-combined constrained Markov decision process (BC-CMDP) framework, which integrates logic-based Benders decomposition and safe reinforcement learning. A natural policy gradient primal-dual optimization is developed to handle the nonconvex policy optimization within the BC-CMDP. The global non-asymptotic convergence of the BC-CMDP framework is rigorously proven. The proposed framework is validated on the IEEE 118-bus system.