Review of vision-based reinforcement learning for drone navigation

Abstract

In recent years, Unmanned aerial vehicles (UAVs) have witnessed a surge in popularity and implementation for both civilian and military usage. UAVs can be utilized for a wide range of applications, including mapping, surveillance, and inspection. For many of these applications, a high level of autonomy is required. Autonomy refers to the ability to complete missions or tasks without human intervention. Autonomous navigation is an essential element of autonomy, especially in GPS-denied environments where GNSS-based navigation is not reliable. Due to size and weight limitations, many UAVs employ vision-based localization and navigation techniques for GPS-denied environments. Reinforcement Learning (RL) is also increasingly being implemented for robotic applications, including obstacle avoidance, battery management, and navigation. Existing reviews typically focus on either vision-based autonomous navigation of drones or RL navigation for drones in general, but none specifically concentrate on the use of vision-based methods and RL for drone navigation. Moreover, previous reviews have highlighted the use of reinforcement learning based on tasks such as takeoff, landing, and navigation, whereas this review categorizes the use of RL based on the navigation problem and image input types for the RL models as these define the needed hardware and processing capabilities of the system. We define the current challenges and limitations for vision based RL navigation to provide direction for future works. Finally we provide an analysis of the favorable conditions for each category and the possibility of combining multiple categories to overcome the disadvantages of each.