Refining the black-box AI optimization with CMA-ES and ORM in the energy management for fuel cell electric vehicles

Abstract

Fuel cell electric vehicles (FCEVs) represent a significant advancement in zero-emission green mobility. By integrating deep reinforcement learning (DRL) for multi-objective energy management strategies, they unlock substantial potential for efficient and sustainable driving. However, the black-box nature of DRL and the challenges in designing multi-objective reward functions pose optimization difficulties. In this paper, we propose to an adaptive evolutionary framework to enhance DRL-based energy management strategies (EMS) by employing the covariance matrix adaptation evolutionary strategies (CMA-ES) for effective black-box optimization. By implementing an opponent reference mechanism, a self-balanced reward function for multiple optimization targets, including vehicle dynamics, powertrain economy, and more, is constructed in the proposed approach. This allows the system to automatically weigh sub-optimization targets and learn superior energy management behaviour via numerous simulation trajectories. The processor-in-the-loop (PIL) test results demonstrate that the proposed solution responds to adaptive adjustment conditions without violating any safety constraints, reduces energy consumption by at least 18.4%, and greatly improves energy utilization efficiency and safety. It exhibits promising optimality in complex energy management problems and robustness to varying velocity profiles, delivering a significant performance advantage over baseline approaches.