Optimizing microgrid energy management with hybrid energy storage systems using reinforcement learning methods

Abstract

With the growth of global energy demand and the pursuit of sustainable energy, microgrids, as an emerging energy supply system, are becoming increasingly important. However, the energy management of microgrid hybrid energy storage systems face numerous challenges, including significant energy waste and poor power supply stability. This study aims to optimize the energy management of microgrid hybrid energy storage systems using reinforcement learning methods. By constructing a reinforcement learning model architecture based on the Markov decision process, the state space, action space, and reward function are systematically designed. The improved proximal policy optimization (PPO) algorithm is then used for implementation. Historical microgrid operation data spanning one year was preprocessed to normalize critical variables, and a simulation was run in a Python environment using OpenAI Gym and proprietary energy system dynamics. The experiment utilizes the operational data of a regional microgrid for one year to compare the traditional model, based on fixed-priority energy allocation rules, with the neural network model. The results show that the reinforcement learning model has an average annual energy management efficiency of 84.5 %, which is significantly improved compared with the 54.25 % of the traditional model and 70 % of the neural network model; the energy loss rate is only 8 %, which is much lower than the 25 % of the traditional model and 18 % of the neural network model; the comprehensive index of power supply stability is 0.92, which is also better than other models. This study provides an efficient and adaptable solution for microgrid energy management, which is expected to promote the healthy development of the microgrid industry.