Reinforcement Learning-Based Control of Voltage-Forming Grid Inverters With Arbitrary Loads

Abstract

Reinforcement learning-based control allows a model-free, self-adaptive control approach with the possibility of nonlinear and complex control, which is desired for inverter-based energy grids, but faces safety-related challenges. To avoid overcurrents and overvoltages in a control task on component level, a safeguard is presented, which is capable of dealing with arbitrary loads without the need of a priori knowledge during the training process of the reinforcement learning-based controller. The control plant is a three-phase, four-wire, three-level 250 kVA-capable inverter with an LC-filter. After 6 min of training, a reinforcement learning controller achieves a normalized mean absolute error (w.r.t. the reference voltage amplitude) in a voltage control task of 0.33% in steady state. During a transient loadstep of 1 kA within 1 ms, corresponding to a current change of 200 % of the rated current, the reinforcement learning controller is capable of safely and stably tracking the setpoint. In a comparison to various model-based, expert-driven controllers, in particular a cascaded PI controller, a predictive dead-beat controller and a combination of both, the reinforcement learning controller shows comparable or better results, both in steady state as well as in transient behavior and enables a highly automated controller design toolchain with minimal human effort, without prior system knowledge and without simulative pretraining.