AI-aided Automated AR-Assisted Assembly Instruction Authoring and Generation method

Abstract

While Augmented Reality (AR) offers the potential to provide real-time guidance, one of the barriers to its adoption in industrial assembly is the lack of fast, no-code, intelligent methods for generating AR-assisted assembly programs. This paper proposes an AI-aided AR-Assisted Assembly Instruction Authoring and Generation method (ARAIAG) to address these challenges. ARAIAG allows engineers, without coding expertise, to intuitively design AR-assisted assembly instructions based on assembly demonstrations captured through RGBD cameras. Based on ARAIAG, we propose a new algorithm considering hand manipulation and model characteristics to achieve spatial registration for models, virtual-physical fusion, and assembly direction recognition. Additionally, we employed a novel human–computer interaction method and Large Language Model (LLM)-assisted content generation to achieve the automatic creation of interactive and instructive AR-assisted assembly programs. Through this approach, we streamline program development and enable more efficient AR-assisted assembly in dynamic manufacturing environments.