Online Hybrid Learning to Speed Up Deep Reinforcement Learning Method for Commercial Aircraft Control

Abstract

We propose an online hybrid learning algorithm that enables deep reinforcement learning agents to learn in environments where the cost of exploration is expensive. Our algorithm adopts ideas from imitation learning and Deep Deterministic Policy Gradient (DDPG). It utilizes an existing baseline controller to speed up the process of learning as well as lower the exploration cost. Our algorithm is validated on classic pendulum swing-up problem and shows faster convergence speed and lower exploration cost. Furthermore, the algorithm can also be applied in learning a controller for commercial aircraft cruising. While DDPG fails to learn a decent policy, our hybrid learning algorithm is able to learn quickly in an online manner with low cost. Our experiments show that the learned policy network is more robust than the baseline PID controller.