A multi-agent reinforcement learning-based method for server energy efficiency optimization combining DVFS and dynamic fan control

With the rapid development of the digital economy and intelligent industry, the energy consumption of data centers (DCs) has increased significantly. Various optimization methods are proposed to improve the energy efficiency of servers in DCs. However, existing solutions usually adopt model-based heuristics and best practices to select operations, which are not universally applicable. Moreover, existing works primarily focus on the optimization methods for individual components, with a lack of work on the joint optimization of multiple components. Therefore, we propose a multi-agent reinforcement learning-based method, named MRDF, combining DVFS and dynamic fan control to achieve a trade-off between power consumption and performance while satisfying thermal constraints. MRDF is model-free and learns by continuously interacting with the real server without prior knowledge. To enhance the stability of MRDF in dynamic environments, we design a data-driven baseline comparison method to evaluate the actual contribution of a single agent to the global reward. In addition, an improved Q-learning is proposed to deal with the large state and action space of the multi-core server. We implement MRDF on a Huawei Taishan 200 server and verify the effectiveness by running benchmarks. Experimental results show that the proposed method improves energy efficiency by an average of 3.9% compared to the best baseline solution, while flexibly adapting to different thermal constraints.