Vehicles-digital twins matching scheme in vehicular edge computing networks: A hierarchical DRL approach

Digital twin (DT) provides a powerful framework to enable various intelligence applications in vehicular edge computing networks. DT servers are used to model and optimize the resource allocation of the whole dynamic system, to provide low latency services for the vehicles. However, the dynamic topology and varying network status make it a challenge to construct the DT model timely, especially in the urban traffic scenario, both the base station (BS) and roadside unit (RSU) cover the ground vehicles overlapped. In this paper, we propose an optimal vehicles-DT servers matching scheme in urban road networks with respect to the dynamic topology and time-varying network status, the DT model selection, building, synchronization and migration latencies are analyzed and optimized mainly. To deal with the complex non-convex problem, we propose a hierarchical reinforcement learning-based solution scheme. The formulated joint optimization problem is decomposed into two subproblems: DT model building and migration. We solve these two subproblems orderly, an improved hierarchical deep reinforcement learning (HDRL)-based algorithm is proposed to find the final optimal solutions. Numerical results demonstrate the convergence of the proposed algorithms, and the effectiveness of the proposed schemes in reducing the system latency.