Multivariable Control of Wastewater Treatment Process Based on Multi-Agent Deep Reinforcement Learning

This paper investigates the multivariable control of wastewater treatment processes (WWTP). This paper integrates deep reinforcement learning (DRL) with PID control and proposes a multivariable adaptive PID control strategy based on multi-agent DRL (MADRL) for WWTP. The approach begins with the construction of a MADRL-PID controller structure, consisting of an agent and a PID controller module. The agent adjusts the PID controller values while the PID module calculates the control signal. To enhance the agent's ability to cooperatively tune multiple PID controllers, the algorithm's components–reward function, action space, environment, and state space–are designed according to the BSM1 simulation platform principles and the MADRL framework requirements. Additionally, to handle WWTP's non-linearities, uncertainties, and parameter coupling, the multi-agent deep deterministic policy gradient algorithm is selected as the foundation for training the agents. Experimental results demonstrate that the proposed algorithm exhibits greater adaptability than traditional PID control and achieves superior control performance.