A novel reinforcement learning-based multi-objective energy management system for multi-energy microgrids integrating electrical, hydrogen, and thermal elements

Abstract

This paper presents a Reinforcement Learning (RL)-driven multi-objective function-based energy management system (RL-MOF-EMS) for optimizing the economic dispatch and lifespan of electrical, thermal, and hydrogen systems within a multi-energy microgrid (MEMG). By leveraging a discrete Deep Q-Network (DQN) agent, the RL-MOF-EMS dynamically balances energy distribution across multiple resources, ensuring optimal thermal and electrical power allocation. The system evaluates three distinct single-objective functions under different EMS strategies: 1) priority-based regulator (PBR), 2) proportional regulator (PR), and 3) particle swarm optimization (PSO). These objectives are then combined into a complex multi-objective optimization problem, solved using the RL-based DQN agent, which selects from 8 dynamic actions to enhance learning speed and accuracy. This RL approach not only accelerates decision-making but also ensures robust and real-time optimization, driving the efficient operation of MEMG systems. The performance is rigorously tested in Simulink under diverse weather conditions and fluctuating thermal and electrical demand profiles. The results are compared against an EMS based on a nonlinear MATLAB optimizer, demonstrating the effectiveness of the RL-MOF-EMS in coordinating power flows from energy sources and storages. The proposed RL-EMS outperforms the FM-EMS by achieving lower operational costs (0.704 €/h vs. 0.767 €/h) and heating costs (1.327 €/h vs. 1.484 €/h), while maintaining a higher hydrogen utilization rate (71.43 % vs. 63.15 %) and state of charge (60.83 % vs. 58.30 %). Additionally, RL-EMS demonstrates superior multi-objective balancing, with a lower overall MOF value and improved performance across key objectives, including operational efficiency, degradation minimization, and heating cost reduction.