O-RAN-Based Digital Twin Function Virtualization for Sustainable IoV Service Response: An Asynchronous Hierarchical Reinforcement Learning Approach

Abstract

Digital Twin for Vehicular Networks (DTVN) continuously simulates and optimizes vehicle behaviors to support emerging 6G Internet-of-Vehicle (IoV) applications such as DT-assisted autonomous driving. To meet Quality of Service (QoS), resource scheduling for distributed vehicle DTs is carried out. However, existing works mainly respond to service demand based on one-to-one DT synchronization and computation offloading, which limits the service response quality and is not sustainable. Meanwhile, twin objects need to be frequently transferred at edges in parallel with the moving vehicles, the IoV service demand response under high-dynamic DT resource distribution is challenging. In this paper, a novel digital twin function virtualization (DTFV) architecture based on Open Radio Access Networks (O-RAN) is proposed. In DTFV, multiple vehicle DTs following one-to-one synchronization are decoupled and reorganized as a Virtualized Digital Twin (VDT) following dissemination-based synchronization for dynamic service response, without needs for offloading service to additional edge devices. Besides, to optimize the overall IoV service response profit, we propose an asynchronous hierarchical reinforcement learning (AHRL)-based DTFV resource scheduling scheme to find optimal VDT orchestration and synchronization strategies. Finally, experimental results show our scheme achieves 8.48% higher service response profit and 6.8% lower VDT synchronization delay over the best baseline scheme.