Physics-shielded deep reinforcement learning for safe energy management of microgrids with battery health consideration

Abstract

While deep reinforcement learning (DRL) algorithms have shown promise in solving the energy management problem in the microgrid (MG), the operational safety of electrical components involved in energy management is often ignored. In this work, a safe DRL-based energy management algorithm is proposed to achieve the supply-demand balance under the high penetration level of wind energy, by optimally coordinating adjustable loads, battery energy storage systems (BESS), and interactions with the main grid. Uniquely, an adaptive physics-shielded mechanism is integrated into the twin delayed deep deterministic policy gradient (TD3) algorithm to enhance the safe operation of the BESS and thus extend its life span. In particular, the state of charge (SoC) is sustained at a safe operating range via the proposed physics-shielded mechanism, and the threshold of the safe range is dynamically adjusted in view of the state of health (SoH). The proposed approach considers both calendar aging and cycling aging to enhance the long-term performance of BESS. Comparisons in real-world dataset show the proposed method can dynamically prevent the SoC of batteries from exceeding the thresholds and substantially extend battery lifespan, in the presence of sharp wind power fluctuation.