Towards Gesture-based Cooperation with Cargo Handling Unmanned Aerial Vehicles: A Conceptual Approach

Abstract

This work provides the fundament for a gesture-based interaction system between cargo-handling unmanned aerial vehicles (UAVs) and ground personnel. The system has platforms with increasing levels of automation in mind and enables operators to visually communicate commands with higher abstractions through a minimum number of necessary gestures. The UAV supports the operator in monitoring surroundings and provides visual feedback in order to increase safety while carrying cargo near ground-level. The interaction concept intends to transfer two goal-directed control techniques to a cargo-handling use case: Object selection via deictic pointing and a proxy manipulation gesture are used to visually communicate intention and control the UAV’s flight. A visual processing pipeline to realize this challenge is presented along with first simulated evaluations of subcomponents.