Efficient active flow control strategy for confined square cylinder wake using deep learning-based surrogate model and reinforcement learning

Abstract

This study introduces a deep learning surrogate model-based reinforcement learning (DL–MBRL) for active control of two-dimensional (2D) wake flow past a square cylinder confined between parallel walls using antiphase jets. In the training of this framework, a proximal policy optimisation (PPO) reinforcement learning agent alternates its interaction between a deep learning-based surrogate model (DL–SM) and a computational fluid dynamics (CFD) simulation to suppress wake vortex shedding, thereby significantly reducing computational costs. The DL–SM, built with a Transformer for temporal dynamics and a multiscale enhanced super-resolution generative adversarial network (MS–ESRGAN) for spatial reconstruction, is trained on 2D direct numerical simulation wake flow data to effectively and accurately emulate complex nonlinear flow behaviours. Compared to standard model-free reinforcement learning, the DL–MBRL approach reduces training time by about 50% while maintaining or improving wake stabilisation. Specifically, it achieves approximately a 98% reduction in shedding energy and a 95% reduction in the standard deviation of the lift coefficient, demonstrating strong suppression of vortex shedding. By leveraging the inherent stochasticity of DL–SM, DL–MBRL also addresses the nonzero mean lift coefficient issue observed in model-free methods, promoting more robust exploration. These results highlight the potential of the framework for extension to practical and industrial flow control problems.