Reinforcement learning based automatic tuning of PID controllers in multivariable grinding mill circuits

Abstract

Process controllers are extensively utilised in industry and necessitate precise tuning to ensure optimal performance. While tuning controllers through the basic trial-and-error method is possible, this approach typically leads to suboptimal results unless performed by an expert. This study investigates the use of reinforcement learning (RL) for the automatic tuning of proportional–integral–derivative (PID) controllers that control a grinding mill circuit represented by a multivariable nonlinear plant model which was verified using industrial data. By employing the proximal policy optimisation (PPO) algorithm, the RL agent adjusts the controller parameters to enhance closed-loop performance. The problem is formulated to maximise a reward function specifically designed to achieve the desired controller performance. Agent actions are analytically constrained to minimise the risk of closed-loop instability and unsafe behaviours during training. The simulation results indicate that the automatically tuned controller outperforms the manually tuned controller in setpoint tracking. The proposed approach presents a promising solution for real-time controller tuning in industrial processes, potentially increasing productivity and product quality while reducing the need for manual intervention. This research contributes to the field by establishing a robust framework for applying RL in process control, designing effective reward functions, constraining the agent to a safe operational space, and demonstrating its potential to address the challenges associated with PID controller tuning in grinding mill circuits.