Optimal control of HVAC systems through active disturbance rejection control-assisted reinforcement learning

Optimal control of multi-zone HVAC systems may suffer from noise and disturbances that affect control accuracy and performance, and faces computational challenges caused by multiple control variables. To address these challenges, this paper proposes a novel method that incorporates reinforcement learning and active disturbance rejection control through a main-auxiliary controller configuration. A main controller is designed based on twin delayed deep deterministic policy gradient, which is responsible for controlling zone supply airflows. An auxiliary controller is configured based on active disturbance rejection control, which regulates the fresh air ratio and meanwhile handling the disturbances and uncertainties. The two controllers work in parallel with exchange information in real-time to optimize HVAC systems in dynamically uncertain environments. In the proposed method, the control variables are separated and handled by main and auxiliary controllers respectively, which reduces the action space of reinforcement learning algorithm and partly decouples the thermal loads and ventilation loads. An EnergyPlus-Python co-simulation platform has been developed using real-world data. Test results demonstrate that the proposed AD-RL method can enhance indoor comfort and IAQ. Furthermore, compared to the rule-based method and the classical TD3-based approach, it can reduce the daily HVAC energy consumption by up to 22.37 % and 13.53 %, respectively.