Challenges in Applying Deep Learning to Augmented Reality for Manufacturing

Abstract

Augmented Reality (AR) for industry has become a significant research area because of its potential benefits for operators and factories. AR tools could help to collect data, create standardized representations of industrial procedures, guide operators in real-time during operations, assess factory efficiency, and elaborate personalized training and coaching systems. However, AR is not yet widely deployed in industries, and this is due to several factors: hardware, software, user acceptance, and companies’ constraints. One of the causes we have identified in our factory is the poor user experience when using AR assistance software. We argue that adding computer vision and deep learning (DL) algorithms into AR assistance software could improve the quality of interactions with the user, handle dynamic environments, and facilitate AR adoption. We conduct a preliminary experiment aiming to perform 3D pose estimation of a boiler with MobileNetv2 in an uncontrolled industrial environment. This experiment produces insufficient results that cannot be directly used but allow us to establish a list of challenges and perspectives for future work.