Reinforcement learning applied to wastewater treatment process control optimization: Approaches, challenges, and path forward

Abstract

Abstract Wastewater treatment process control optimization is a complex task in a highly nonlinear environment. Reinforcement learning (RL) is a machine learning technique that stands out for its ability to perform better than human operators for certain high-dimensional, complex decision-making problems, making it an ideal candidate for wastewater treatment process control optimization. However, while RL control optimization strategies have shown potential to provide operational cost savings and effluent quality improvements, RL has proven slow to be adopted among environmental engineers. This review provides an overview of existing RL applications for wastewater treatment control optimization found in literature and evaluates five key challenges that must be addressed prior to widespread adoption: practical RL implementation, managing data, integrating existing process models, building trust in empirical control strategies, and bridging gaps in professional training. Finally, this review discusses potential paths forward to addressing each key challenge, including leveraging soft sensing to improve online data collection, working with process engineers to integrate RL programming with existing industry software, utilizing supervised training to build expert knowledge into the RL agent, and focusing research efforts on known scenarios such as the Benchmark Simulation Model No. 1 to build a robust database of RL agent control optimization results. GRAPHICAL ABSTRACT