Design and Evaluation of a Mixed Reality-based Human-Robot Interface for Teleoperation of Omnidirectional Aerial Vehicles

Abstract

Omnidirectional aerial vehicles are an attractive solution for visual inspection tasks that require observations from different views. However, the decisional autonomy of modern robots is limited. Therefore, human input is often necessary to safely explore complex industrial environments. Existing teleoperation tools rely on on-board camera views or 3D renderings of the environment to improve situational awareness. Mixed-Reality (MR) offers an exciting alternative, allowing the user to perceive and control the robot’s motion in the physical world. Furthermore, since MR technology is not limited by the hardware constraints of standard teleoperation interfaces, like haptic devices or joysticks, it allows us to explore new reference generation and user feedback methodologies. In this work, we investigate the potential of MR in teleoperating 6DoF aerial robots by designing a holographic user interface (see Fig. 1) to control their translational velocity and orientation. A user study with 13 participants is performed to assess the proposed approach. The evaluation confirms the effectiveness and intuitiveness of our methodology, independent of prior user experience with aerial vehicles or MR. However, prior familiarity with MR improves task completion time. The results also highlight limitation to line-of-sight operation at distances where relevant details in the physical environment can still be visually distinguished.