A robust safe reinforcement learning-based operation method for hybrid electric-hydrogen energy system risk-based dispatch considering dynamic efficiency characteristics of electrolysers

Abstract

Hybrid electric-hydrogen energy systems hold transformative potential in achieving significant green energy transitions by leveraging complementary storage and flexibility. To safeguard operation against the variability of large-scale renewable generation, this paper formulates a risk-based dispatch for such systems that explicitly models the dynamic efficiency of electrolyzers. We propose a robust Soft Actor-Critic algorithm grounded in deep reinforcement learning to solve the resulting nonconvex, nonlinear, stochastic scheduling problem online, without resorting to simplifying approximations. A robust constrained Markov decision process framework is developed, which interprets constraint violations as an exploratory cost and uses the conditional value at risk of that cost to enforce a risk-averse policy. A novel second-order Bellman operator efficiently estimates this risk metric, while a primal-dual optimization scheme ensures maximum-entropy learning under safety constraints. Case studies on modified IEEE-118 and South Carolina 500-bus systems demonstrate that our approach converges 35.5 % faster and maintains superior constraint satisfaction compared to state-of-the-art deep reinforcement learning methods. Against traditional optimization-based methods, it reduces expected overloads by 21.9 %, peak overloads by 43.8 %, and improves overall computational efficiency by 99.994 %.