Off-policy reinforcement learning control for space manipulators based on object detection via convolutional neural networks

Abstract

This paper proposes a vision-based control framework that integrates convolutional neural network-based object detection with off-policy reinforcement learning to address the engineering demands of autonomy, robustness, and high control performance in space manipulator operations, as well as to fill gaps in existing vision-based control research. A two-loop architecture comprising a detection loop and a control loop is constructed, with a combined-variable approach employed to simplify the complex image-space dynamics of the space manipulator. On the vision side, a state-of-the-art single-stage object detection network is enhanced with a depth regression module to provide real-time distance feedback. On the control side, an off-policy reinforcement learning algorithm is adopted to achieve model-free optimal control. The proposed integrated vision-based control strategy is validated through both verification and comparative simulations, demonstrating superior autonomy, robustness, and control performance, as well as advantages over the other representative vision-based control method.