Optimizing radiant floor heating control with night setbacks using model-free reinforcement learning and transfer learning

Abstract

Controlling Radiant Floor Heating (RFH) systems with night setbacks presents a challenge due to their slow-response dynamics. Model Predictive Control (MPC) has demonstrated effectiveness in controlling such systems, but the need for model development constrains its scalability. This study investigates the feasibility and approaches of using model-free Reinforcement Learning (RL) and transfer learning for optimal control of RFH systems with night setbacks. A physics-based model is developed and validated as a virtual testbed. Four distinct RL control (RLC) strategies are proposed and evaluated, alongside a conventional Rule-Based Control (RBC) strategy as a baseline, and an MPC as an upper-bound performance benchmark. Our findings reveal that the Deep Q-Network (DQN) with n-step Temporal Difference learning incorporating weather forecasts as states achieve the best performance. The heating demand is reduced by 15-23 % with RLC and 13.1-28.5 % with MPC against RBC. However, the unmet hours with RLC are higher than those of MPC, suggesting further research for constraint satisfaction improvement. The transferability of the RLC is also evaluated by applying the trained RL agent to a new building using transfer learning through weights initialization, layer freezing and fine tuning. The results show that the training time for a new agent with a target building is significantly reduced taking advantage of transfer learning from an existing agent trained with a source building. In conclusion, this study demonstrates the potential of model-free RL and transfer learning in optimizing RFH systems with night setbacks, fostering advancements in scalable optimal building control strategies.