UAV autonomous obstacle avoidance via causal reinforcement learning

Abstract

The role of unmanned aerial vehicles (UAVs) in everyday life is becoming increasingly important, and there is a growing demand for UAVs to autonomously perform obstacle avoidance and navigation tasks. Traditional UAV navigation methods typically divide the navigation problem into three stages: perception, mapping, and path planning. However, this approach significantly increases processing delays, causing UAVs to lose their agility advantage. In this paper, we propose a causal reinforcement learning-based end-to-end navigation strategy that directly learns from data, bypassing the explicit mapping and planning steps, thus enhancing responsiveness. To address the issue where using a continuous action space prevents the agent from learning effective experiences from past actions, we introduce an Actor–Critic method with a fixed horizontal plane and a discretized action space. This approach enhances the efficiency of sampling from the experience replay buffer and stabilizes the optimization process, ultimately improving the success rate of the reinforcement learning algorithm in UAV obstacle avoidance and navigation tasks. Furthermore, to overcome the limited generalization capability of end-to-end methods, we incorporate causal inference into the reinforcement learning training process. This step mitigates overfitting caused by insufficient interaction with the environment during training, thereby increasing the success rate of UAVs in performing obstacle avoidance and navigation tasks in unfamiliar environments. We validate the effectiveness of causal inference in improving the generalization capability of the reinforcement learning algorithm by using convergence steps in the training environment and navigation success rates of random targets in the testing environment as quantitative metrics. The results demonstrate that causal inference can effectively reduce overfitting of the policy network to the training environment.