Immersive remote grasping: realtime gripper control by a heterogenous robot control system

Current developments in the field of user interface (UI) technologies as well as robotic systems provide enormous potential to reshape the future of human-robot interaction (HRI) and collaboration. However, the design of reliable, intuitive and comfortable user interfaces is a challenging task. In this paper, we focus on one important aspect of such interfaces, i.e., teleoperation. We explain how to setup a heterogeneous, extendible and immersive system for controlling a distant robotic system via the network. Therefore, we exploit current technologies from the area of virtual reality (VR) and the Unity3D game engine in order to provide natural user interfaces for teleoperation. Regarding robot control, we use the well-known robot operating system (ROS) and apply its freely available modular components. The contribution of this work lies in the implementation of a flexible immersive grasping control system using a network layer (ROSbridge) between Unity3D and ROS for arbitary robotic hardware.