Research on Switching Strategy with Reinforcement Learning and Game Theory in Satellite-Terrestrial Integrated Networks

Abstract

In Satellite-Terrestrial Integrated Networks (STIN), from the perspective of increasing the capacity of the networks, the user experience, and the adaptability to high-speed motion occasions, a non-cooperative multi-service network selection scheme based on Q-learning and game theory (QRSG) is proposed. QRSG first obtains the multi-service network utility through the fuzzy process and uses it as the reward of Q-learning. The state of Q-learning includes the quality of service (QoS) and price attributes of the network currently connected by the user, as well as the situation of the user speed. The corresponding network selection strategy is the action of Q-learning. Then, the user predicts the payoff of the network selection strategy through a game algorithm to avoid access to an overloaded network. In addition, Binary Exponential Backoff Algorithm is introduced in QRSG to solve the problem of inaccurate throughput prediction in the scenario where multiple users concurrently switch to the same service node (SN). Simulations reveal that: 1) With QRSG, users with different speeds and QoS requirements can adaptively switch to the most suitable network. 2) Compared with the existing algorithms, QRSG can increase network throughput by more than 8% and reduce the total number of switching by about 60% in the case of a maximum loss of 1 to 2% of the system fairness.