Human robot interaction for agricultural Tele-Operation, using virtual Reality: A feasibility study

Abstract

With the increasing demand for efficient and sustainable agricultural practices, the automation of tasks such as crop inspection and harvesting has become a critical endeavor. However, the complex and dynamic nature of agricultural environments poses challenges for conventional methods that are fully autonomous or those relying on traditional interfaces. To address these challenges, we propose a solution that leverages the capabilities of Virtual Reality (VR) to provide operators with an intuitive and immersive control experience. This paper introduces a novel method for tele-operating a robotic system in agriculture using VR technology. By integrating a VR device with SteamVR and Unity 3D, users can control a mobile robotic module over a local network or the internet using VR hand controllers and a headset. In order to validates the system feasibility, we case studied two agricultural operations in lab settings: leaf inspection and crop harvesting.

The results of this study were evaluated based on the cycle completion time (CCT) and the success rate of robot-plant interaction (RPI). For fruit harvesting, with a sample size (N) = 5, the mean CCT was approximately 18 s, with a standard deviation of nearly 5 s, indicating an improvement compared to existing autonomous systems in the literature. Additionally, in the leaf inspections, the mean CCT resulted in approximately 26 s with the standard deviation of nearly 6 s with the same sample size. The RPI success rate reached up to 90 % in the fruit harvesting practices. And in leaf inspection practices, this metric averaged two attempts per diseased leaf, 50 %, to grasp it and bring it to the operator’s attention. Through this study, the combination of consumer-grade VR technologies with a mobile robotic manipulation system highlights the system’s promise in improving remote agricultural tasks, especially in response to labor scarcity and improving farmworker efficiency.