Deep Reinforcement Learning Based Double-layer Optimization Method for Energy Management of Microgrid

Abstract

Microgrid provides an effective way to integrate renewable energy into power grid. However, the uncertainty of renewable energy and load demand bring great challenges to the energy management of microgrid. Therefore, this paper proposes a double-layer optimization method based on deep reinforcement learning (DRL) to solve this problem. The upper DRL agent takes Soft actor-critic algorithm to fully explore the regulation ability of the energy storage system. The lower nonlinear programming solver optimizes the output of other controllable equipment based on the output of the upper layer, and constantly revises the network parameters of the upper layer according to the optimization results. By combining DRL with traditional nonlinear programming, the convergence speed of the algorithm can be improved and the design difficulty of the DRL reward function can be reduced. Case studies show that the double-layer collaborative optimization method can provide real-time highquality solutions for energy management of the microgrid only based on the immediate information of the microgrid and can effectively accelerate the convergence speed of the model.