Dual Network Computation Offloading Based on DRL for Satellite-Terrestrial Integrated Networks

Abstract

Satellite-terrestrial integrated networks based on edge computing can provide computation offloading service to terminal devices in remote areas. However, it faces various limitations, including satellite energy consumption, computation delay, and environmental dynamics, etc. In this paper, we propose a satellite-terrestrial integrated cloud and edge computing network (STCECN) architecture, including satellite layer, terrestrial layer and cloud center, where computing resources exist in multi-layer heterogeneous edge computing clusters. Optimization of system delay and energy consumption is defined as a mixed-integer programming problem. Moreover, we present a deep reinforcement learning-based computation offloading decision algorithm that can adapt to the dynamics and variability of satellite networks. A dual network computation offloading decision method is proposed for delay and energy consumption based on deep reinforcement learning offloading (DRLO), including deep convolutional network update method, quantization strategy, and bandwidth resource allocation. Meanwhile, the proposed method is based on previous experience and integrates deviation adjustment strategies for decision making to solve the problem of pseudo-patch loss caused by satellite network switching. The simulation results indicate that the proposed method performs almost consistently with traditional heuristic algorithms, with only 20% of the time consumption of the latter, and the number of pseudo packet loss also decreases to the original 10–20%.