Sim-to-real design and development of reinforcement learning-based energy management strategies for fuel cell electric vehicles

The application of reinforcement learning (RL) algorithms in energy management strategies (EMSs) for fuel cell electric vehicles (FCEVs) has shown promising results in simulations. However, transitioning these strategies to real-vehicle implementation remains challenging due to the complexities of vehicle dynamics and system integration. Based on the General Optimal control Problems Solver (GOPS) platform, this paper establishes an RL-based EMS development toolchain that integrates advanced algorithms with high-fidelity vehicle models, leveraging Python-MATLAB/Simulink co-simulation for agent training across Model-in-the-Loop (MiL), Hardware-in-the-Loop (HiL), and Vehicle-in-the-Loop (ViL) stages. Besides, the distributional soft actor-critic algorithm (DSAC) is applied to energy management for the first time, embedding the return distribution function into maximum entropy RL. This approach adapts the Q-value function update step size, significantly enhancing strategy performance. Additionally, two RL-based EMS frameworks are investigated: one where the agent directly outputs fuel cell power commands, and another where the agent generates equivalent factors (EF) for the equivalent consumption minimization strategy (ECMS). Simulation and experimental results validate that both RL frameworks achieve superior fuel economy, reducing hydrogen consumption by approximately 4.35 % to 5.73 % compared to benchmarks. By combining Python's algorithmic flexibility and scalability with MATLAB/Simulink's high-fidelity vehicle models, the proposed toolchain provides a robust foundation for real-vehicle applications of RL-based EMSs.