Learning-to-Fly RL: Reinforcement Learning-based Collision Avoidance for Scalable Urban Air Mobility

As hundreds of Unmanned Aircraft System (UAS) operate within urban airspaces, automated and decentralized UAS traffic management (UTM) will be critical to maintain safe and efficient operations. In this work, we present Learning-to-Fly with Reinforcement Learning (L2F-RL), a decentralized, on-demand Collision Avoidance (CA) framework that systematically combines machine learning with cooperative model predictive control for UAS collision avoidance while retaining satisfaction of higher-level mission objectives. L2F-RL consists of: 1) RL-based policy for conflict resolution (CR) with discrete-decision making, 2) decentralized, cooperative model predictive control for CA. To accelerate training with RL, we utilize reward shaping and curriculum learning. Our approach outperforms baseline approaches with a 99.10% separation rate (ratio of success to total test cases) in the worst case, improving to 100% in the best case with a 1000X improvement in computation time compared to centralized methods. Our results demonstrate the potential of combining learning approaches with optimization-based control, making it a significant contribution towards scalable, decentralized UTM.