Federated heterogeneous multi-agent deep reinforcement learning-based attack resilience scheduling for heterogeneous multi-integrated energy system

Abstract

The increasing scale of heterogeneous multi-integrated energy system (MIES) exposes critical limitations in conventional scheduling methods, particularly regarding privacy preservation, coordination of heterogeneous systems, and resilience against cyberattacks. To address these challenges, this paper proposes a resilient scheduling framework based on federated heterogeneous multi-agent soft actor-critic (Fed-HMASAC), integrating federated learning (FL) and heterogeneous multi-agent deep reinforcement learning (HMADRL). Firstly, a heterogeneous MIES model incorporating multi-energy coupling and a dynamic price attack (DPA) mechanism is established. Furthermore, a federated heterogeneous multi-agent architecture is developed, which coordinates the collaboration of differentiated state/action space agents based on the advantage function decomposition, and proposes an adversarial training mechanism to enhance the resilience of the policy network against DPAs. The experimental results show that the proposed framework possesses better economic performance compared to the baseline approach while maintaining the stability of the scheduling strategies under continuous DPA conditions, and achieves data privacy preservation through FL.