Taming deep reinforcement learning-based conflict resolution in air traffic control using geometric technique

Recent advances in Deep Reinforcement Learning (DRL)-based CR have shown promise; however, the end-to-end nature of DRL systems, which rely on reward-driven mechanisms, poses challenges in adhering to ATC’s stringent regulations. Traditional geometric methods, such as the Modified Voltage Potential (MVP) technique, theoretically meet ATC requirements by minimizing deviations from planned paths during conflicts but struggle to ensure optimal performance in complex, uncertain environments. In this study, we amalgamate DRL with the geometric MVP technique, leveraging DRL’s capacity for intelligent decision-making in complex environments and MVP’s ability to theoretically minimize deviations from planned paths. Within this framework, we utilize a look-ahead DRL agent, in conjunction with MVP conflict detection methods, to foresee potential conflicts. Should a conflict be imminent, a Maneuver DRL agent takes immediate control of the aircraft to adeptly navigate the situation, ensuring conflict avoidance. Once the aircraft is clear of the conflict, a rule-based method is employed to swiftly return it to the planned path. Extensive simulations demonstrate that our proposed framework significantly reduces conflict rates, maintains efficient trajectory adherence with minimal deviations, lowers unnecessary computational overhead, and effectively adapts to dynamic environmental conditions.