Cooperative energy management and fuel cell thermal management of fuel cell hybrid electric buses via multi-agent reinforcement learning

Abstract

This study proposes a collaborative control strategy based on multi-agent reinforcement learning (MARL) to tackle the dynamic coupling optimization challenge between the energy management strategy (EMS) and the fuel cell thermal management strategy (FCTMS) in hydrogen fuel cell hybrid electric buses. Specifically, the EMS adopts a multi-objective equivalent consumption minimization strategy (ECMS), incorporating penalty terms for powertrain state-of-health (SOH) degradation and temperature violations. To maintain battery state-of-charge (SOC), this multi-objective ECMS employs the twin delayed deep deterministic policy gradient (TD3) algorithm to dynamically adjust its equivalent factor (EF). Meanwhile, the FCTMS utilizes fuel cell power as one of the state information to guide another TD3 agent in synchronously optimizing coolant flow rate and radiator air flow rate, ensuring optimal operating temperature. Results show that, compared with a hierarchical control strategy integrating ECMS (neglecting powertrain durability and thermal characteristics) and PID-based FCTMS, the proposed strategy significantly reduces driving cost, fuel cell SOH degradation, battery SOH degradation, fuel cell temperature violations, fuel cell outlet–inlet temperature difference violations and battery temperature violations by 5.30%, 3.86%, 10.62%, 99.02%, 47.68% and 29.33%, respectively. Moreover, further investigation demonstrates that the proposed strategy exhibits superior adaptability to unknown driving scenarios.