Carbon-Aware Decision-Making for Data Centers in Intraday Electricity Markets Using Deep Reinforcement Learning

Abstract

With the growing demand for computational services, data centers (DCs) have become major electricity consumers, contributing significantly to carbon emissions and posing new challenges to power system flexibility and sustainability. This study addresses the intraday scheduling of DCs under high-frequency fluctuations caused by renewable integration and load uncertainty. A carbon-aware bi-level optimization framework is proposed: the upper level clears the market based on electricity prices and carbon intensity, while the lower level minimizes the total operating cost, including electricity procurement, carbon emissions, and task migration costs. To overcome the real-time challenges of intraday decision-making, the problem is formulated as a Markov Decision Process (MDP), and a deep reinforcement learning (DRL) framework based on Proximal Policy Optimization (PPO) is developed. The agent interacts with the environment to dynamically adjust power procurement and task scheduling. The convergence and training stability of the proposed method are validated through representative learning metrics, including mean reward, explained variance, total loss, and value loss. Simulation results on a modified IEEE 5-bus system show that the approach outperforms traditional day-ahead strategies, reducing costs and emissions while enhancing spatiotemporal coordination. The PPO algorithm also exhibits a better performance than other benchmark DRL methods. Overall, the DRL-based carbon-aware scheduling strategy offers promising potential for advancing low-carbon and resilient power systems.