Novel energy management strategy for fuel cell/battery hybrid energy systems combining MPC and deep reinforcement learning

Abstract

This paper proposes a novel energy management strategy (EMS) for fuel cell/battery hybrid energy systems by integrating model predictive control (MPC) with deep reinforcement learning (DRL).The proposed EMS leverages the advantages of both MPC and DRL, effectively addressing MPC’s performance degradation due to model uncertainties, while simultaneously accelerating DRL convergence and enhancing its adaptability to unforeseen conditions. Specifically, the study first formulates a dynamic model of the fuel cell/battery hybrid energy system, incorporating component degradation characteristics. Based on this, the corresponding MPC model is then developed. MPC serves as the baseline controller, ensuring system stability and constraint adherence through a linearized model, while DRL provides a compensatory policy to enhance the system’s long-term decision-making capability. The combined control strategy is applied to optimize the hybrid energy system, with objectives carefully designed to balance state of charge (SOC) maintenance, hydrogen consumption, and degradation costs of each energy source. Simulation results demonstrate that the proposed control strategy outperforms both the standalone MPC-based EMS and the DRL-based EMS across multiple performance indicators. Compared to MPC, the proposed strategy results in a 4.41 % increase in battery degradation but achieves a significant 51.43 % reduction in fuel cell degradation. Moreover, while maintaining battery SOC, it achieves the lowest system operating cost, reducing it by 4.45 % and 2.13 % compared to MPC and DRL, respectively. Furthermore, comparative analyses with classical EMSs and validations under unknown scenarios further highlight the robustness and overall performance advantages of the proposed strategy.