Deep reinforcement learning for HVAC control with nonlinear parametric thermal network modeling for passive building envelopes

The incorporation of passive cooling envelopes into buildings can effectively reduce energy consumption. However, due to their limited cooling capacity, HVAC systems are still required to maintain indoor thermal comfort. In buildings using passive radiative cooling roofs and thermochromic windows as passive cooling envelopes, inappropriate HVAC control strategies are more likely to occur due to the changeable optical and thermal properties. Such improper HVAC control results in significant waste during system operation, which remains an unsolved problem. Therefore, optimizing HVAC operation in passively cooled buildings is essential not only for ensuring thermal comfort but also for further reducing energy consumption. Achieving both objectives depends on effectively capturing the building's thermal behavior and efficient control methods. To incorporate the thermal behavior of passive cooling envelopes into the HVAC control system, this study first develops a resistance-capacitance thermal network based on a modified matrix to predict the thermal behavior of passively cooled buildings. Then, a model-based policy optimization deep reinforcement learning (DRL) control method is proposed to enhance HVAC system performance in such buildings. The results show that the modified matrix significantly improves the prediction accuracy of the thermal behavior of passively cooled buildings compared to current global identification methods, which increases coefficient of determination from 0.90, 0.73, 0.88 to 0.94, 0.92, 0.95 for radiative cooling roofs, thermochromic windows, and a combination of both envelopes, respectively. Moreover, the proposed DRL control method can reduce building energy consumption by 17.7 % for radiative cooling roofs, 10.6 % for thermochromic windows, and 21.1 % when both strategies are applied simultaneously, compared to the baseline control method. This study provides valuable insights into optimization of HVAC system operations in buildings equipped with passive cooling envelopes.