Magic Mirror Interface for Augmented Reality Maintenance: An Automotive Case Study

We present a novel interaction method for augmented industrial maintenance based on a "magic mirror" interface and virtual motion buttons. The system includes a video camera for object tracking, a video\depth camera for capturing user gestures, a projector for displaying technical instruction to the operator and a LCD monitor providing feedback of the virtual buttons. The operator can trigger maintenance commands by directional swift of the hands in regions sensitive to motion speed and direction. The main advantage of the presented interface is that it can work in realistic industrial conditions: (i) operators wearing gloves, (ii) operators handling tools, (iii) presence of moving machinery and personnel in the background. We measured the performances of the system with a laboratory test and we proved the feasibility with an automotive inspection test case. We calculated an average interaction time below 2 seconds and an error rate lower than 5%. However, we found some performances limitations if the operator is handling tools.