A scalable demand-side energy management control strategy for large residential districts based on an attention-driven multi-agent DRL approach

Abstract

The growing penetration of renewable energy sources holds great potential for decarbonizing the building energy sector. However, the intermittent and unpredictable nature of renewable generation poses significant challenges to grid stability and energy integration. Demand-side management (DSM) has emerged as a promising solution, leveraging demand flexibility to align energy consumption with periods of peak renewable generation and mitigate grid instability. To fully harness this flexibility, energy coordination across multiple buildings is essential, enabling participation in flexibility markets and optimizing energy management at district level. This paper introduces attention-actor-critic multi-agent deep reinforcement learning (AAC-MADRL), an actor-critic algorithm built upon the centralized training with decentralized execution (CTDE) framework, enhanced with attention mechanisms with the aim of enabling scalable, coordinated, and autonomous DSM in residential districts. A parameterized reward structure allows systematic testing under different cooperation scenarios – fully cooperative, competitive, and mixed – highlighting the conditions where AAC-MADRL outperforms other deep reinforcement learning (DRL) approaches, including decentralized and non-attention-based cooperative methods. Evaluated through winter and summer scenarios in districts across Alameda County, California (73 buildings) and Texas County (100 buildings) using the CityLearn platform, AAC-MADRL demonstrates substantial improvements. AAC-MADRL achieves energy cost reductions of up to 18 % in Texas and 12.5 % in California compared to the rule-based controller. Additionally, it improves self-sufficiency by 6 %–10.5 % during periods of limited solar generation and significantly reduces peak demand. The algorithm also exhibited superior computational efficiency, with deployment times 40.5 % faster than decentralized DRL and 62.5 % faster than cooperative non-attention-based DRL approaches on average.