Reinforcement learning for data center energy efficiency optimization: A systematic literature review and research roadmap

Abstract

With today’s challenges posed by climate change, global attention is increasingly focused on reducing energy consumption within sustainable communities. As significant energy consumers, data centers represent a crucial area for research in energy efficiency optimization. To address this issue, various algorithms have been employed to develop sophisticated solutions for data center systems. Recently, Reinforcement Learning (RL) and its advanced counterpart, Deep Reinforcement Learning (DRL), have demonstrated promising potential in improving data center energy efficiency. However, a comprehensive review of the deployment of these algorithms remains limited. In this systematic review, we explore the application of RL/DRL algorithms for optimizing data center energy efficiency, with a focus on optimizing the operation of cooling systems and Information and Communication Technology (ICT) processes, including task scheduling, resource allocation, virtual machine (VM) consolidation/placement, and network traffic control. Following the Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) protocol, we provide a detailed overview of the methodologies and objectives of 65 identified studies, along with an in-depth analysis of their energy-related results. We also summarize key aspects of these studies, including benchmark comparisons, experimental setups, datasets, and implementation platforms. Additionally, we present a structured qualitative comparison of the Markov Decision Process (MDP) elements for joint optimization studies. Our findings highlight vital research gaps, including the lack of real-time validation for developed algorithms and the absence of multi-scale standardized metrics for reporting energy efficiency improvements. Furthermore, we propose joint optimization of multi-system objectives as a promising direction for future research.