Research on Intuitive Gesture Recognition Control and Navigation System of UAV

Abstract

UAVs (unmanned aerial vehicles) are mostly equipped with GPS and additional sensors. However, the control of the UAV still depends on the skill of the operator. Inexperienced control often leads to accidents that damage the UAV and harm the environment, pedestrians, and buildings. In this research, we propose a superior control of the UAV through intuitive gestures based on deep learning of gesture recognition, which reduces the difficulty of UAV control. To improve the flight control technology of the UAV and reduce accidents caused by improper UAV control, an intuitive gesture recognition control system is constructed. The gesture recognition control is operated by a series of gesture recognition and LSTM (Long Short-Term Memory) neural networks which output the label as the control commands of the UAV. Eight different control commands are defined and generated for the control. After identifying the pickup gesture, the coordinates of the index finger are projected to the UAV's screen, and the target can be easily positioned to identify objects. The system is expected to be used for subsequent automatic flight navigation. The gesture recognition system achieves 99.54% accuracy in the training set and 99.17% accuracy in the testing set. The method to realize the control of the UAV is to send a control command back to the UAV after the computer recognizes a frame of the input hand image. The original output screen of gesture recognition has only 10‒12 FPS, and the control of the drone has a latency of about 83.33‒100 ms. After using multi-threaded processing, the FPS is increased to 15‒16, which reduces the delay, so that the latency is only about 62.5‒66.67 ms. Through a high-accuracy and low-latency intuitive gesture recognition control system, we have enough confidence to replace the method of controlling UAVs with remote control for easy control.