Systematic AR-based assembly guidance for small-scale, high-density industrial components

Abstract

Small-scale high-density industrial components (SHIC) are widely used in high-end electromechanical equipment with higher requirements for functional integration. SHIC assembly operation is a typical manual fine operation, which is labor-intensive, time-consuming, and error-prone and seriously affects assembly efficiency and quality consistency. The current augmented reality (AR) technology cannot realize the complete, accurate, and stable cognition and guidance for SHIC because of the single use of visual cognition. Meanwhile, manual preparation of prior knowledge dramatically reduces AR applications’ automation level and practicability. Addressing this research gap, this paper develops a technical framework of AR-based assembly guidance for SHIC, which integrates several novel intelligent methods and deeply combines prior knowledge reasoning and computer vision cognition in a simple and interpretable way. First, computer vision and deep learning are used to automatically generate prior knowledge from the three-dimensional (3D) model of SHIC. Then, 3D tracking, visual recognition, virtual-real matching, and rule reasoning are combined to complete, sequence, and locate the assembly targets in the case of insufficient visual information. Finally, based on the adaptive threshold, two kinds of guidance modes, i.e., position-based precise guidance and region-based heuristic guidance, are automatically switched to realize the AR guidance of the whole assembly process. The case study of complex electrical connectors shows that this framework with novel intelligent methods can meet the industrial requirements of performance, availability, and effectiveness and improve the robustness and efficiency of SHIC assembly operation by 92.95% and 87.06%, respectively. This study realizes AR guidance technology’s systematic and practical application in SHIC assembly operation. Also, it provides a practicable technology architecture for intelligent AR assistance in the fine assembly field.