SRAD: Autonomous Decision-Making Method for UAV Based on Safety Reinforcement Learning

Abstract

Unmanned aerial vehicles (UAVs) are increasingly vital across numerous sectors, from logistics and rescue operations to military endeavours and beyond. However, ensuring safety in the decision-making processes surrounding UAV operations in real-world settings has become an urgent and complex challenge. At present, the main methods to minimise the risk of drone decision-making include utilising pre-established control rules, expert prior knowledge and regularisation constraints. However, these methodologies require UAVs to meet demanding prerequisites, including the acquisition of extensive decision-making experience and the establishment of comprehensive rules. Regrettably, these strict requirements often lead to frequent UAV crashes in uncertain environments and subsequent mission failures. In order to tackle these issues, we propose a self-decision-making method for quadcopter UAVs based on safe reinforcement learning. Our method utilises a multilevel cascading feature semantic space for reinforcement learning, integrating depth images, greyscale images, semantic segmentation images and object detection results as inputs. This approach aims to facilitate safe autonomous learning. Moreover, we integrate real offline labelled data to enhance the safety policy. Depending on the varying levels of risk encountered during the UAV's decision-making process, we dynamically select different safety policies. Through this iterative process, the UAV progressively eliminates extreme actions and reverts to the UAV learning policy module. Experimental results indicate that our method not only ensures safe decision-making for UAVs in uncertain environments but also exhibits superior safety decision-making efficacy compared to certain baseline methods.