Fuel- and noise-minimal departure trajectory using deep reinforcement learning with aircraft dynamics and topography constraints

Abstract

Designing an optimal departure trajectory for an airport can minimize fuel emissions within the surrounding airspace and noise perceived by nearby populations, which brings positive sociological and economic implications in addition to environmental benefits. Yet, designing a trajectory that considers realistic operational constraints could be complex and, consequently, computationally expensive. Traditional trajectory optimization methods often simplify the problem to manage computational costs, which leads to compromised accuracy. To overcome this challenge, we propose a reinforcement learning (RL) approach that can satisfy multidisciplinary constraints by leveraging accurately modeled flight dynamics, high-fidelity population data, and topological data. This is achieved by establishing a comprehensive, physically-consistent simulated environment for the learning algorithm, while keeping the computational cost low. Instead of directly designing the trajectory itself, we train an RL agent to control the aircraft, whose trajectory is then considered as optimal. We model the RL problem as a continuous Markov decision process and employ the soft actor-critic architecture. By changing the relative importance of fuel consumption and noise in the optimization objective, we can obtain different optimum trajectories that are well-suited to the specific region of interest. Not surprisingly, a trade-off between fuel consumption and noise impact is observed in our results. This developed framework provides a more accurate and sophisticated approach for departure trajectory optimization, whose results are beneficial for future airspace design and can support sustainable aviation efforts.