Model-free Reinforcement Learning for Demand Response in PV-rich Distribution Systems

Abstract

The recent government initiatives to decarbonize the power system and minimize the overreliance on fossil fuels will lead to massive and rapid adoption of solar photovoltaics (PVs) in distribution systems. Daytime solar power generation peaks necessitate demand-side flexibility to mitigate voltage dips and spikes that could disrupt the power grid. To this end, a model-free demand response framework is developed based on deep reinforcement learning (DRL) and using OpenAI Gym APIs. The DRL agent assumes the role of a load aggregator that directly controls a percentage of each load in the distribution system. The agent is then trained to optimize the control policy by taking nonuniform actions on each node. The system-wide objective is to minimize the voltage deviations from 3% of the nominal voltage in an effort to properly allocate the energy consumption throughout the day. The DRL-based demand response is trained and tested on the radial IEEE 33-bus distribution feeder, modified to have a high penetration of non-dispatchable PVs plants. Simulation results show that the proposed framework works as intended, shifting flexible demand to times of maximum solar power generation while maintaining an acceptable voltage deviation at each node.