A reinforcement learning-based control approach with lightweight feature for robotic fish heading control in complex environments: Real-world training

Abstract

In complex and unpredictable ocean environments, reinforcement learning (RL) based intelligent control strategy holds significant potential for enhancing the survival and operation performance of robotic fish. Although it is well known that training RL-based control strategy in real-world environments possesses numerous advantages, yet achieving it remains significant challenges such as training approach and computational power. This paper proposes a RL-based lightweight control strategy for robotic fish heading direction control, which can be implemented on a compact control system (72 MHz) in the physical world. The control strategy integrates RL with a central pattern generator (CPG). Specifically, RL is employed for decision-making processes, whereas CPG plays a crucial role in action execution. Functional segregation within this framework enables low computational consumption while facilitating rapid convergence. The proposed control strategy is validated through the conduction of diverse directional swimming tests in a wire-driven robotic fish that is trained in a circulating water channel. The complex flow environment and the damaged caudal fin are incorporated into the tests, demonstrating the satisfactory generalization performance, robustness, and adaptability to unknown control scenarios in practical applications. The proposed control strategy exhibits approximately a $50\%$ enhancement in control accuracy and a notable improvement in stability compared to the traditional PID approach. This study offers a valuable reference for the development and implementation of RL-based control strategy in robotic fish, specifically for physical training and directional control during swimming.