Navigation of Sounding Balloons with Deep Reinforcement Learning

Abstract

With the availability of high performance miniaturized electronics, sounding balloons have become a viable options to conduct scientific experiments and commercial missions in the stratosphere, acting as a reduced size, low mass, low cost alternative to large zero-pressure or superpressure balloons. This paper explores the use of deep reinforcement learning for controlling a stratospheric sounding balloon to perform station-keeping over a specified area. In particular, we implement the deep Q-network (DQN) algorithm to learn a control policy for the balloon by exploiting different wind directions at different altitudes, reached by dropping ballast or releasing lifting gas. We conduct experiments using a simulation environment and evaluate the performance of the trained DQN model in real historical data. Our results show that the DQN algorithm can effectively learn a control policy that achieves satisfactory station-keeping with a high success rate, outperforming other, more direct control approaches. Our study presents a possible solution for the control of stratospheric sounding balloons in various applications.