A self-optimizing defrost initiation controller for air-source heat pumps: Experimental validation of deep reinforcement learning

Abstract

Air-source heat pumps (ASHPs) play a key role in sustainable heating, but their efficiency is significantly reduced by frost formation on the evaporator. The timing of defrost initiation is crucial to minimize energy losses, yet conventional demand-based defrosting (DBD) controllers rely on specialized sensors for frost detection and heuristic thresholds for defrost initiation, leading to increased system costs and suboptimal performance. This paper presents an experimental validation of a self-optimizing deep reinforcement learning (RL) controller. With our proposed implementation, RL determines defrost timing using standard temperature measurements and autonomously generates tailored control rules, overcoming the limitations of conventional DBD methods. The study consists of three case studies conducted on a hardware-in-the-loop test bench with a variable-speed ASHP. First, RL’s defrost timing accuracy is evaluated against experimentally pre-determined optima. Across five stationary test conditions, RL achieves near-optimal defrost initiations with maximum efficiency losses of at most 1.9 %. Second, RL is benchmarked against time-based (TBD) and demand-based defrost controllers for three typical days with varying ambient conditions. RL outperforms TBD by up to 7.1 % in $SCOP$ and 3.6 % in heat output. Compared to DBD, RL improves $SCOP$ by up to 9.1 % and heat output by 4.9 %. Finally, we assess RL’s ability to adapt its strategy through online learning. We emulate airflow blockage, a common soft-fault condition, caused by obstructions on the evaporator fins (e.g., leaves). RL adjusts its strategy to the changed environment and improves efficiency by 16.6 %. While the results are promising, limitations remain, requiring further research to validate RL in real-world ASHPs.