All-in-One: Unified Computing and Networking Resource Scheduling for Next-Generation Converging Networks

The emerging intelligent services, spurred by the rise of the intelligent Internet, are placing multidimensional requirements on the network to collaboratively guarantee computing and networking resources. In this article, we propose a unified end-to-end intelligent resource scheduling method for converging networks [e.g., Internet of Things (IoT)], which can always globally abstract the available resources from different networks with a unified model description, and jointly planning the resources from end-to-end by deep reinforcement learning (DRL) algorithms supporting both discrete and continuous variable decisions. The method proposes a three-layer architecture, including service layer, network layer, and adaption layer, which aims at optimizing the flow transmission performance. Through the general Markov decision process (MDP) transformation from the model, the DRL-assisted algorithm can further solve the optimization problem. We categorize heterogeneous network resource scheduling into horizontal and vertical scenarios, applying the proposed architecture to both. Compared with the existing diverse learning (DiLearn) and naive (DiNaive) approaches, the proposed approach is not only time-saving but also can schedule 28.4% and $8\times $ more flows in horizontal scheduling scenarios, and improve 54.2% and $3.5\times $ flows in vertical scheduling scenarios, respectively.