Intra-day dispatch method via deep reinforcement learning based on pre-training and expert knowledge

Traditional economic dispatch algorithms rely on the accuracy of all parameters and also lack the adaptability to the high uncertainties brought by the dynamic changes happening in the current power systems. Its computing efficiency also needs to be improved with the increased operational complexities. In recent years, due to high self-learning and self-optimization ability, reinforcement learning has emerged in the field of economic dispatch, which can solve model-free dynamic programming problems that cannot be effectively solved by traditional optimization methods. In this paper, we construct a reinforcement agent for intra-day dispatch to optimize generator output, using a twin delayed deep deterministic policy gradient algorithm based on pre-training and expert knowledge (PEK-TD3). Aiming at solving the problems of long exploration time and poor convergence of conventional deep reinforcement learning, we propose an initial policy network training method based on pre-training with supervised learning, which significantly speeds up the training process of deep reinforcement learning and greatly reduces the model development cycle. At the same time, expert knowledge is embedded in the deep reinforcement learning to guide the training of the agent. With the guidance of expert knowledge, on the one hand, the agent quickly learns to limit the search direction to the feasible region of the power system operation so as to improve the convergence. On the other hand, in order to obtain higher rewards, agent learns to prioritize the renewable energy utilization which significantly reduces the curtailment rate of renewable energy. Finally, the modify IEEE 118-node system is used to verify the performance of the proposed method.