Fast peg-in-hole assembly policy for robots based on experience fusion proximal optimization

Background: As an important part of robot operation, peg-in-hole assembly has problems such as a low degree of automation, a large amount of tasks and low efficiency. It is still a huge challenge for robots to automatically complete assembly tasks because the traditional assembly control policy requires complex analysis of the contact model and it is difficult to build the contact model. The deep reinforcement learning method does not require the establishment of complex contact models, but the long training time and low data utilization efficiency make the training costs very high. Methods: With the aim of addressing the problem of how to accurately obtain the assembly policy and improve the data utilization rate of the robot in the peg-in-hole assembly, we propose the Experience Fusion Proximal Policy Optimization algorithm (EFPPO) based on the Proximal Policy Optimization algorithm (PPO). The algorithm improves the assembly speed and the utilization efficiency of training data by combining force control policy and adding a memory buffer, respectively. Results: We build a single-axis hole assembly system based on the UR5e robotic arm and six-dimensional force sensor in the CoppeliaSim simulation environment to effectively realize the prediction of the assembly environment. Compared with the traditional Deep Deterministic Policy Gradient algorithm (DDPG) and PPO algorithm, the peg-in-hole assembly success rate reaches 100% and the data utilization rate is 125% higher than that of the PPO algorithm. Conclusions: The EFPPO algorithm has a high exploration efficiency. While improving the assembly speed and training speed, the EFPPO algorithm achieves smooth assembly and accurate prediction of the assembly environment.