Enhancing offline reinforcement learning for wastewater treatment via transition filter and prioritized approximation loss

Abstract

Wastewater treatment plays a crucial role in urban society, requiring efficient control strategies to optimize its performance. In this paper, we propose an enhanced offline reinforcement learning (RL) approach for wastewater treatment. Our algorithm improves the learning process. It uses a transition filter to sort out low-performance transitions and employs prioritized approximation loss to achieve prioritized experience replay with uniformly sampled loss. Additionally, the variational autoencoder is introduced to address the problem of distribution shift in offline RL. The proposed approach is evaluated on a nonlinear system and wastewater treatment simulation platform, demonstrating its effectiveness in achieving optimal control. The contributions of this paper include the development of an improved offline RL algorithm for wastewater treatment and the integration of transition filtering and prioritized approximation loss. Evaluation results demonstrate that the proposed algorithm achieves lower tracking error and cost.