CCS-MASAC Resource Allocation Method for Collaborative Cluster Satellite Systems in 6G

Abstract

The collaborative cluster satellite system (CCS) within the 6G network establishes the foundation for robust services in the future Star-Earth integrated network by coordinating multiple low-earth orbit (LEO) satellites for collaborative observation missions and efficient space mission processing. This article proposes a model-based soft actor-critic (SAC) algorithm, CCS-MASAC, for optimizing throughput in clustered satellite systems within 6G networks. The algorithm integrates the clustering degree of CCS with the entropy regularization term in SAC, proposing an adaptive adjustment method. Unlike existing studies, in this work, we adopt an environment model-based policy optimization approach for the first time. Model-based policy optimization focuses on improving the sample efficiency of reinforcement learning (RL) algorithms. It allows agents to learn iteratively in both real and simulated environments, which improves sample efficiency, convergence, and algorithm robustness. To address the dimensionality explosion in single-agent RL algorithms, we extend this approach to a multiagent RL algorithm by defining observable neighborhoods for each agent, further enhancing performance. Simulation results indicate that the CCS-MASAC algorithm proposed in this article enhances throughput by 15%–20% and accelerates convergence by 30% compared to existing algorithms, including the multiagent deep Q-network (MADQN), multiagent proximal policy optimization (MAPPO), multiagent deep deterministic policy gradient (MADDPG) and multiagent double and dueling deep Q-learning (MAD3QL). The scalability and robustness of the algorithms are verified by scalability experiments and experiments under dynamic channel conditions. This research provides new solutions for throughput optimization and resource management in CCS systems.