End-to-end deep reinforcement learning and control with multimodal perception for planetary robotic dual peg-in-hole assembly

The planetary construction is necessary for long-term scientific deep space exploration and resource utilization in the future. The planetary robotic assembly control is a key technology that must be broken through in future planetary surface construction. The paper focuses on the most representative dual peg-in–hole assembly, which has sufficiently complex contact interaction, wide range of applications and good method portability. To address the challenges brought by the unstructured planetary environment and the features of the construction tasks, the paper proposes an end-to-end deep reinforcement learning and control method with multimodal perception for planetary robotic assembly tasks. A staged reward function based on the visual virtual target point for policy learning is designed. The effectiveness and feasibility of the proposed control method have been verified through simulation experiments and ground real robot experiments. It provides a feasible control method of robotic operations for future planetary surface construction.