Automatic 3D inspection method for AR-assisted assembly based on virtual-to-real registration

Abstract

Augmented reality (AR) has been widely employed in intelligent assembly tasks to improve assembly efficiency by layering virtual instructions onto real assemblies, providing operators with step-by-step guidance. However, current AR-assisted assembly systems are limited to being visualization tools, requiring manual control of the guide program by operators and potentially causing distractions and increased operational load. Furthermore, these systems lack the ability to detect incorrect assembly during operation, leading to assembly failures without manual inspection. To address these issues, we propose an automatic 3D inspection method based on virtual-to-real registration that leverages cross-domain texture registration and 6D pose registration to align real assembly images with virtual 3D CAD models. This method conducts a 3D assembly inspection by assessing the similarities between real assembly and its virtual CAD instruction, not only from texture but also from spatial pose relations, improving inspection accuracy while retaining 2D real-time computing. By integrating the inspection results, the AR system can automatically verify assembly correctness and proceed to the next guide program only when a successful assembly is confirmed, eliminating any need for extra instructions from the operator. In case of assembly failure, the computed results are fed back to the operator to assist in correcting errors during assembly, thereby improving assembly efficiency.