Deep Reinforcement Learning-Based Beam Hopping Scheduling Algorithm for Satellite Internet System

Satellite internet is a new network system that combines satellite communication and internet technology. The main space-based infrastructure of satellite internet is a low-Earth-orbit (LEO) satellite constellation. The system uses flexible control of satellite antenna pointing to provide worldwide on-demand coverage with finite satellites’ resource. beam hopping (BH) is a promising technology to improve resource allocation's flexibility, which typically uses spot beams to illuminate different beam positions in different time slots. However, considering the uneven and dynamic traffic demand of every cell, it is difficult to design a dynamic beam hopping scheduling (BHS) algorithm that can provide high resource utilisation and good Quality of Experience (QoE). We propose a deep reinforcement learning (DRL)-based downlink BHS algorithm, which leverages DRL to make dynamic decisions according to varying states. The experimental results show that compared with other algorithms, the proposed algorithm performs better in terms of average queuing delay, capacity utilisation ratio, fairness between different cells and the whole throughput.