Q-Learning for Sum-Throughput Optimization in Wireless Visible-Light UAV Networks

Unmanned aerial vehicles (UAVs) have been adopted as aerial base stations (ABSs) to provide wireless connectivity to ground users in events of increased network demand, and points-of-failure infrastructure (such as in disasters). However, with the existing crowded radio frequency (RF) spectrum, UAV ABSs cannot provide high-data-rate communication required in 5G and beyond. To address this challenge, visible light communication (VLC) is proposed to be equipped on UAVs to take advantage of the flexible and on-demand deployment feature of the UAV, and the high-data-rate communication of the VLC. However, VLC has strong alignment requirements between transceivers, therefore, how to determine the position and orientation of the UAV is critically important for sum-throughput improvement. In this paper, we propose two Q-learning based methods to maximize the sum throughput of the wireless visible-light UAV network by jointly controlling the position and orientation of the UAV. The results show that the proposed approaches can achieve a network-wide data rate very close to the optimal solution obtained by exhaustive search and outperform up to 18% compared with the intuitive centroid-based method. Computation complexity is also evaluated, where results showing that the proposed two Q-learning based methods can both consume less computational time, i.e., approximately 9 times and 210 times less on average than that of the exhaustive search approach.