Adaptive transfer reinforcement learning (TRL) for cooling water systems with uniform agent design and multi-agent coordination

Abstract

Transfer Reinforcement Learning (TRL) offers a promising approach to optimizing building cooling water systems by improving both energy efficiency and operational effectiveness. This study introduces a novel TRL framework designed to accelerate the learning process of Reinforcement Learning (RL) agents by systematically leveraging prior experience from analogous systems. Unlike conventional RL approaches that start from scratch, our framework enhances initial performance while mitigating negative transfer through an adaptive multi-agent supervision mechanism. Our methodology involves three key innovations. First, we collect field data from three cooling water systems and train individual data-driven models to enable realistic energy and control simulations. Next, we design a uniform RL model with ratio-based inputs and outputs, ensuring transferability across systems with varying characteristics. This model includes two controllers—one for cooling towers and the other for cooling water pumps. Third, we introduce a structured TRL process in which a pre-trained RL model from a source system is transferred to two target systems. Within this transfer framework, we integrate three types of agents: one completely new agent and two variants of the trained RL model. A key feature of the framework is a supervision mechanism that coordinates these agents for positive transfer. It dynamically adjusts the selection probability of each agent through a constantly updated preference function and modifies learning objectives at different training stages. Various scenarios are tested to evaluate the framework’s performance with different transferred agents and learning stages. Simulation results demonstrate a 10 % improvement in energy savings—both initially and at convergence—compared to learning from scratch. Moreover, the proposed TRL framework effectively mitigates negative transfer and avoids converging to the suboptimal performance of a transferred agent. More importantly, it significantly reduces the effort required to select appropriate source systems, highlighting its practical applicability and potential for widespread adoption in building cooling water system optimization.