Deep reinforcement learning-based multi-objective optimization for electricity–gas–heat integrated energy systems

Abstract

With the increasing global attention on energy efficiency and carbon emissions, the optimization of integrated energy systems (IES) has become the key to improve energy efficiency and reduce pollution emissions. However, most of the existing optimization methods cannot effectively deal with the complexity of high dimensional continuous action space. Therefore, this paper focuses on a novel multi-objective optimization strategy for the electricity–gas–heat integrated energy systems (EGH-IES). Firstly, considering the absorption capacity of wind power and the emission of pollutant gases, a multi-objective optimization model is constructed based on the mechanism model and operation constraints of each device in EGH-IES, in which the integrated operation cost and the environmental factors are taken as optimization objectives. Then, the multi-objective optimization problem is designed as the optimal strategy of interaction learning between agent and environment in reinforcement learning, and the output power of the devices constitutes the action of reinforcement learning. Additionally, the Ornstein–Uhlenbeck process is introduced to enhance the training efficiency and exploration performance of the agent, and the deep deterministic policy gradients (DDPG) algorithm is employed to optimize the action, thus the output power of the appliances could be obtained. Finally, the simulation results show that compared with deep Q network (DQN) method and proximal policy optimization (PPO) method, the reward function value of the proposed method increases by 2.43% and 6.09%, respectively, which represents a reduction in economic cost and pollutant emissions. These verify the effectiveness and superiority of the proposed multi-objective optimization scheme in cost reduction and benefit improvement for the EGH-IES.