Inspection of mechanical assemblies based on 3D deep learning segmentation

We are focused on conformity control of complex aeronautical mechanical assemblies, typically an aircraft engine at the end or in the middle of the assembly process. Our overall system should ensure that all the mechanical parts are present and well-mounted. A 3D scanner carried by a robot arm provides acquisitions of 3D point clouds which are further processed. Computer-Aided Design (CAD) model of the mechanical assembly is available. In this paper, we are concentrating on detecting the absence of mechanical elements. Previously we have developed a rendering pipeline for creating realistic synthetic 3D point cloud data. We do this by using the CAD model and taking into account occlusion and self-occlusion of mechanical parts. In this paper, an existing deep neural network for 3D segmentation is experimentally chosen and trained on these synthetic data. Further, the model is evaluated on real data acquired by a 3D scanner and has shown good quantitative results according to a segmentation metric. Finally, when a threshold is applied to the segmentation result, a final decision is made on the absence/presence problem. The achieved accuracy is 98.7%. Our research work is being carried out within the framework of the joint research laboratory ”Inspection 4.0” between IMT Mines Albi/ICA and the company Diota specialized in the development of numerical tools for Industry 4.0. This research is a continuation of the work presented at the QCAV’2021 conference [1].