Hierarchical deep reinforcement learning for dynamic reconfiguration of photovoltaic integration in distribution network

In the context of the dual-carbon strategy for achieving carbon peak and neutrality, the increasing penetration of new energy sources, particularly distributed photovoltaic (PV) systems, has made the distribution network (DN) environment more complex, leading to serious curtailment issues in modern distribution systems. This paper introduces a voltage deviation index into the reward function of deep reinforcement learning (DRL) and proposes a Hierarchical Deep Reinforcement Learning (HDRL) method, which divides the agent in the Reinforcement Learning (RL) algorithm into high-level and low-level controllers to analyze the operating environment of the DN from different spatial and temporal scales, optimizing the proactive strategy for Dynamic Reconfiguration of Distribution Networks (DNDR). The high-level control model is responsible for formulating global strategies and selecting different switching group actions, while the low-level control model is responsible for executing specific switching actions. This approach incentivizes the agent to reduce the selection of switching actions, thus avoiding the algorithm from falling into local optima in complex environments. Case studies show that the method saves $563.34 in total economic cost and improves voltage quality by 91.71 %. The case studies validate that this method can more effectively enhance the consumption capacity of distributed PV, reduce the phenomenon of curtailment, and improve the operational stability and reliability of the DN.