Learning human-like decisions for flight approach: Environment and an imitation learning method

¹ Flight approach in terminal airspace is a challenging task with high aircraft density and maneuvering in air traffic control decisions. Existing reinforcement learning methods were only studied based on simulation environments, and also suffered from sparse reward and state-space explosion problems. In this work, an imitation learning-based autonomous framework, AppGAIL, is proposed to achieve the flight approach decision based on human expert demonstrations, which has the ability to eliminate the requirement of designing handcrafted rewards. To cope with the state-space explosion problem, a cylindrical grid airspace model is designed to convert the earth space to discrete airspace, obtaining the transformation of the real-time traffic situation by near distance–identical cells. The generative adversarial mechanism is applied to achieve imitation learning by distinguishing the source of the input observations (state and action sequences with a sliding window), i.e., from the generator or expert demonstrations. Since all human expert demonstrations are safe operations, limiting the model to learn knowledge of flight conflicts and confusing the generator to plan conflict trajectories, a conflict-aware discriminator is proposed to detect possible conflicts by a multi-task framework with learnable weights, which further supports the adversarial training. The real-world traffic dataset is applied to validate the proposed method, in which several custom metrics are proposed to support the real-world air traffic control. The experimental results demonstrate that the AppGAIL outperforms other baseline methods, achieving only 0.67% potential conflict rate and 3.732 kilometers dynamic time wrapping distance. Most importantly, all proposed technical modules contribute the desired performance improvement. Additionally, multi-aircraft planning and real-time factors can also be resolved to improve the applicability of the proposed method.