SmartARW: A text-aware smart mobile industrial augmented reality (AR) wiring assembly system

Abstract

Augmented reality (AR) has demonstrated its potential by delivering intuitive guidance on the workbench directly, alleviating the operators’ mental load for traditional paper-based wiring assembly tasks while ensuring procedural correctness. Nevertheless, current industrial AR wiring applications mainly focus on superimposing the virtual models with the commercial AR glass, lacking practical adaptability due to ergonomic issues derived from their weight, and human intervention is also necessary to monitor the current wiring harness and activate ongoing procedural progress. To bridge this gap for real-world applications, this article proposes human-centric SmartARW: a text-aware smart mobile industrial AR wiring assembly system. Firstly, a mobile AR wiring assembly system is established with the off-the-shelf tablet and sensor module, followed by accurate system calibration among different components and self-contained markerless motion tracking. Then, a closed-loop confirming strategy enabled lightweight text-aware network is proposed and integrated into the mobile AR system, as well as lots of shop-floor datasets are prepared and augmented to improve the context-aware performance, thus the status of the ongoing wiring activity can be perceived accurately even encountered challenge scenarios while activating the corresponding AR instructions automatically. Finally, extensive quantitative and qualitative experiments are carried out to illustrate that the proposed SmartARW has strong usability and can provide superior performance for human-centric smart wiring assembly action.