Graph Neural Network-based Deep Reinforcement Learning algorithm for Virtual Network Function forwarding graph embedding in Space–Air–Ground Integrated Network

Space–Air–Ground Integrated Network (SAGIN) can provide seamless three-dimensional coverage and enhanced flexibility, and it has been recognized as the core architecture of future 6G networks. Software Defined Network (SDN) and Network Function Virtualization (NFV) are two enabling technologies of SAGIN, which can sequentially arrange multiple Virtual Network Functions (VNFs) into VNF Forwarding Graph (VNF-FG) to provide users with scalable and parallel network services. However, SAGIN exhibits significant dynamism and heterogeneity, and VNFs may be deployed in multiple different heterogeneous locations, which brings great challenges to the efficient embedding of VNF-FGs required for user service request flows. In this paper, we study the VNF-FG embedding problem by jointly considering the structural constraints, node and link resource constraints, and End-to-End (E2E) delay constraint of VNF-FG in SDN/NFV-enabled SAGIN. Specifically, we first design a three-layer SDN/NFV-enabled SAGIN architecture consisting of a global controller and distributed intra-domain SDN controllers. Then, we define the Delay and Cost Efficient Dynamic VNF-FG Embedding Problem (DCE-DVEP) and formulate it as an Integer Linear Programming (ILP) with the objective of minimizing the weighted sum of E2E delay and embedding cost of all VNF-FGs. Finally, a Graph Neural Network-based Deep Reinforcement learning Embedding (GNN-DRE) algorithm is proposed to solve the DCE-DVEP, which can more accurately capture the rich feature information from both SAGIN and VNF-FG by specifically integrating different GNN models and adopts Deep Reinforcement Learning (DRL) to make effective embedding decisions. The simulation results demonstrate that, compared with other baseline algorithms, the GNN-DRE can reduce E2E delay and embedding cost by about 8% and 13%, respectively.