FERO: Efficient Deep Reinforcement Learning based UAV Obstacle Avoidance at the Edge

Abstract

With the expanding use of unmanned aerial vehicles (UAVs) across various fields, efficient obstacle avoidance has become increasingly crucial. This UAV obstacle avoidance can be achieved through deep reinforcement learning (DRL) algorithms deployed directly on-device (i.e., at the edge). However, practical deployment is constrained by high training time and high inference latency. In this paper, we propose methods to improve DRL-based UAV obstacle avoidance efficiency through improving both training efficiency and inference latency. To reduce inference latency, we employ input dimension reduction, streamlining the state representation to enable faster decision-making. For training time reduction, we leverage transfer learning, allowing the obstacle avoidance models to rapidly adapt to new environments without starting from scratch. To show the generalizability of our methods, we applied them to a discrete action space dueling double deep Q-network (D3QN) model and a continuous action space soft actor critic (SAC) model. Inference results are evaluated on both an NVIDIA Jetson Nano edge device and a NVIDIA Jetson Orin Nano edge device and we propose a combined method called FERO which combines state space reduction, transfer learning, and conversion to TensorRT for optimum deployment on NVIDIA Jetson devices. For our individual methods and combined method, we demonstrate reductions in training and inference times with minimal compromise in obstacle avoidance performance.