Q-Learning for Sum-Throughput Optimization in Wireless Visible-Light UAV Networks

Abstract

Unmanned aerial vehicles (UAVs) have been adopted as aerial base stations (ABSs) to provide wireless connectivity to ground users in events of increased network demand, and points-of-failure infrastructure (such as in disasters). However, with the existing crowded radio frequency (RF) spectrum, UAV ABSs cannot provide high-data-rate communication required in 5G and beyond. To address this challenge, visible light communication (VLC) is proposed to be equipped on UAVs to take advantage of the flexible and on-demand deployment feature of the UAV, and the high-data-rate communication of the VLC. However, VLC has strong alignment requirements between transceivers, therefore, how to determine the position and orientation of the UAV is critically important for sum-throughput improvement. In this paper, we propose two Q-learning based methods to maximize the sum throughput of the wireless visible-light UAV network by jointly controlling the position and orientation of the UAV. The results show that the proposed approaches can achieve a network-wide data rate very close to the optimal solution obtained by exhaustive search and outperform up to 18% compared with the intuitive centroid-based method. Computation complexity is also evaluated, where results showing that the proposed two Q-learning based methods can both consume less computational time, i.e., approximately 9 times and 210 times less on average than that of the exhaustive search approach.