Reinforcement Learning Control of Power Systems with Unknown Network Model under Ambient and Forced Oscillations

Abstract

We present a model-free optimal control design for electric power systems with unknown transmission network and load models to improve its dynamic performance using techniques from reinforcement learning (RL) and adaptive dynamic programming (ADP). We consider different persistent disturbances in the grid including ambient oscillations resulting from load fluctuations and their effects on exciter voltage regulation loops. We also consider forced oscillation scenarios that frequently occur due to malfunctioning of governor valves. Our proposed RL algorithm recovers the optimal feedback response in spite of all of these disturbances in a completely model-free way using online measurements of the states, inputs, and the disturbances. The design is validated using the IEEE benchmark 39-bus, 10-generator New England power system model perturbed with different ambient and forced oscillations.