High-Precision Pose Estimation Method of the 3C Parts by Combining 2D and 3D Vision for Robotic Grasping in Assembly Applications

Abstract

Using robots to replace manual production has been seen as a feasible solution to reduce production costs and increase productivity, especially in the 3C (Computer, Communication, and Consumer Electronics) products assembly lines, which rely heavily on labor. Due to the characteristic of small size and accurate fitting precision, small uncertainties in the assembly process will lead to the failure of the assembly, especially in the process of grasping, when the gripper needs to move to a certain position. So, to realize the 3C products flexible assembly of the robot, machine vision is required to provide the high-precision pose information of the object. So far, point cloud based 6D (6-dimensional) pose estimation algorithms have attracted the attention of many researchers because point cloud can provide three-dimensional information directly. However, the disorder of the point cloud and the background information with miscellaneous noise makes it impossible to directly estimate the pose of the target object with high precision. To deal with this problem, we propose a 2D-3D combined high-precision pose estimation method. The whole method is divided into two stages. In the first stage, the mask of the object in the 2D image is identified through the Mask R-CNN which is trained through fine-turning. In the second stage, we use a structured light camera to generate the point cloud and map the mask to it to extract useful point cloud, then the high-precision pose estimating algorithm composed by PCA-ICP is used to get the global pose of the part. Finally, the pose is converted to the robot coordinate frame by the result of hand-eye calibration. The proposed method is verified by the grasping and assembly experiments.