Enhanced energy management and cyber-resilience of integrated energy systems based on optimal energy storage system and demand response: A deep reinforcement learning approach

Abstract

The integrated multi-carrier energy system (IMCES) optimal operation based on dynamic variables is complex and challenging. This study proposes a framework for multi-level energy management in the IMCES structure. The framework takes into account demand response (DR) programs, effective participation of energy storage systems, and presents the cyberspace structure's resilience against cyber-attacks such as false data injection (FDI). The framework improves the resilience by using a robust multi-agent deep reinforcement learning (RMADRL) strategy. The multi-objective function has been formulated in three levels. The first-level objectives are to optimize the operation of IMCES through the DR program and to optimize the participation of ESS units through the wholesale and retail market prices. The second-level objectives aim to minimize the cost of greenhouse gas emissions, while the third-level objectives evaluate the cyberspace based on fixed and random cyber attacks. The RMADRL strategy is a method that uses the Markov decision process equations to evaluate optimal actions and policies. The multi-agent soft actor-critic method is utilized for this purpose. By executing the DR program, the total operation cost can be reduced by 9.9%. Furthermore, executing the DR program and incorporating the ESSs effective participation can further reduce the total operation cost by 15.79%.