Digital twin-based pursuit-evasion gaming strategy optimization for underwater robot grasping

Abstract

Underwater robotic grasping challenges are essential for the advancement of underwater robotics and oceanic development. To tackle the difficulties encountered by these robots in grasping, we present an innovative multi-agent learning framework based on a pursuit-evasion game. This framework consists of three phases: initial learning, interactive learning, and independent learning, enabling a gradually enhanced learning experience. We propose a robot pursuit approach utilizing Improved Grey Wolf Optimization (IGWO) and implement the Soft Actor-Critic for learning target evasion strategies. The IGWO augments search and sample methodologies, markedly enhancing search efficacy relative to the conventional Grey Wolf Optimization. Furthermore, we have created virtual reality software for underwater robots and implemented a related digital twin system platform, facilitating the training and education of pursuers and evaders in a simulated environment. Ultimately, we implement this system in a practical underwater pursuit-evasion scenario. Through interactive training and iterative learning, the robotic arm exhibits the capability to strategically pursue an evasive target, while the target demonstrates adaptable escape. Both modeling and experimental results produce excellent outcomes, offering innovative approaches and insights for the dynamic grasping domain of underwater robotics.