Combining Interactive Spatial Augmented Reality with Head-Mounted Display for End-User Collaborative Robot Programming

This paper proposes an intuitive approach for collaborative robot end-user programming using a combination of interactive spatial augmented reality (ISAR) and headmounted display (HMD). It aims to reduce user’s workload and to let the user program the robot faster than in classical approaches (e.g. kinesthetic teaching). The proposed approach, where user is using a mixed-reality HMD – Microsoft HoloLens – and touch-enabled table with SAR projected interface as input devices, is compared to a baseline approach, where robot’s arms and a touch-enabled table are used as input devices. Main advantages of the proposed approach are the possibility to program the collaborative workspace without the presence of the robot, its speed in comparison to the kinesthetic teaching and an ability to quickly visualize learned program instructions, in form of virtual objects, to enhance the users’ orientation within those programs. The approach was evaluated on a set of 20 users using the within-subject experiment design. Evaluation consisted of two pick and place tasks, where users had to start from the scratch as well as to update the existing program. Based on the experiment results, the proposed approach is better in qualitative measures by 33.84% and by 28.46% in quantitative measures over the baseline approach for both tasks.