Resource-constrained dynamic planning and model-free control in turbulent urban environments

Abstract

This work proposes a descriptive digital twin of an urban region with a sufficiently small footprint to run online in resource-constrained quadrotor systems. The digital twin describes the building geometries and turbulent kinetic energy (TKE) within an urban region, enabled via our proposed surrogate model (SM) for TKE reconstruction. This forms the basis of a simulation environment for autonomous path following and collision avoidance using Deep Reinforcement Learning (DRL). A Voronoi-based turbulence-weighted graph (TWG) is developed for safe path planning and is capable of reacting to dynamic changes in wind direction and, consequently, the turbulence field. Lastly, the environment simulates oncoming traffic of dynamic unknown obstacles for a vision-enabled DRL agent to evade. Several DRL agents with different observation and action spaces are trained and evaluated.

The SM enables rapid reconstruction of the turbulence with an accuracy comparable to full-order methods. The TWG plans safe paths that reduce the worst-case TKE exposure by 44% at the cost of increasing the average path length by 33% compared to a shortest-distance approach. The DRL agents successfully solve the navigation problem with a 100% success rate in the static obstacle scenario, where the minimum clearance between buildings is $1.0m$. In scenarios with dynamic obstacles, the agent achieves a 79% success rate. Suggestions for further performance and safety improvements in the TWG planner and DRL agents are presented.