Physics-Constrained Reinforcement Learning for Energy Management in Fuel-Cell Hybrid UAVs

Abstract

Fuel-cell hybrid UAVs require energy management strategies to maintain efficiency during rapid load transients. Existing methods often neglect the fuel cell's limited dynamic response, leading to frequent and abrupt power adjustments. To address this, we propose a physics-constrained energy management strategy based on the soft actor–critic (SAC) algorithm. A minimum fuel-cell power holding time and a maximum allowable power change are embedded into the control environment to ensure that the generated fuel-cell power commands satisfy the prescribed supervisory constraints. The proposed strategy is evaluated under multiple flight scenarios and compared with fuzzy logic (FL), equivalent consumption minimization strategy (ECMS), and constrained dynamic programming (DP). The results show that SAC achieves a hydrogen consumption of 5.4679 g, only 0.32% higher than DP, while maintaining a final SOC of 59.67%. Its equivalent hydrogen consumption is 5.4811 g, which is 0.79% and 1.32% lower than those of ECMS and FL, respectively. Moreover, SAC achieves a fuel-cell power STD of 91.38 W, representing reductions of 41.82% and 47.05% compared with ECMS and FL, respectively, while the RMS of the fuel-cell power change rate is 25.29 W·s⁻¹. These results demonstrate that the proposed strategy provides well-balanced performance in terms of hydrogen consumption, battery SOC regulation, and fuel-cell power regulation.