Deploying deep reinforcement learning for low-level HVAC control in multi-zone buildings: A comparative study with ASHRAE G36 sequences

Abstract

This paper proposes a methodology for optimizing HVAC control in multi-zone buildings using Deep Reinforcement Learning. The study focuses on optimizing the central AHU system by controlling all low-level components within both the air and water loops, addressing the complex dynamics of multi-zone interactions. The case study is based on a building within the Politecnico di Torino campus. Modelica-based simulations are used to model both the HVAC system and building dynamics, allowing the integration and evaluation of the ASHRAE G36 control standard as a benchmark. Two DRL strategies are developed and evaluated, Zone-Aware and Zone-Integrated, under both winter and summer conditions, with the goal of improving energy efficiency, indoor temperature control, and indoor ${\text{CO}}_2$ concentration, under varying occupancy profiles. The results reveal that both DRL strategies outperform the G36 baseline in terms of energy savings (up to 17 %), indoor temperature violations, and ${\text{CO}}_2$ concentration. Additionally, DRL controllers demonstrate strong generalizability and adapt seamlessly to unseen occupancy profiles without manual tuning. This research highlights the potential of DRL to provide scalable, adaptive, and energy-efficient HVAC control solutions for multi-zone buildings.