Optimal inverter-based droop control in active distribution network with evolutionary strategy-embedded deep reinforcement learning

Inverter-based photovoltaics based on a Volt/Var curve can support Voltage/Var control (VVC) and have been widely adopted in active distribution networks. Due to its data-driven characteristics, deep reinforcement learning (DRL) has been widely applied to improve the benefits of the VVC. However, since the traditional default droop control lacks system-wide optimum, a comprehensive droop control function is proposed to improve voltage regulation performance. This work of optimal droop control function is challenging as the Volt/Var curve introduces products of voltage variables, leading to an intractable bilinear optimization problem. Moreover, a decision space too large for conventional DRL leads to a local optimum. Thus, we propose an evolutionary strategy embedding the conventional DRL to enhance exploration and sample efficiency, deriving optimal droop control functions. The proposed method’s efficiency and superiority in power loss reduction and voltage regulation enhancement are verified through case studies involving the modified IEEE 33-bus distribution systems.