Deep Reinforcement Learning-based Volt-Var Optimization in Distribution Grids with Inverter-based Resources

Abstract

High penetration of solar photovoltaic (PV) units in distribution grids and the variability of their output power have caused new challenges to grid operators. Voltage fluctuations and their impact on system losses under high PV penetration scenarios are among these emerging challenges. This paper proposes a deep reinforcement learning based Volt-Var optimization method to minimize voltage fluctuations under high penetration of distributed energy resources, such as PV units, in distribution networks. The Deep deterministic policy gradient (DDPG) approach is developed and used for regulating the voltage in the network while minimizing network losses. The DDPG determines the optimal schedule of reactive power output of PV units and battery energy storage devices through controlling their inverters. The reward function includes both the voltage regulation and power loss minimization objectives. The performance of the proposed approach is validated on a modified version of the IEEE-34 bus system with added PVs and BESs and under various PV penetration scenarios. The results show that both voltage fluctuation and power loss reduces when the agent is fully trained that verify the performance of the proposed model.