Distributed RL-Based Resource Allocation and Task Offloading for Vehicular Edge of Things Computing

Abstract

Smart vehicles are increasingly equipped with advanced sensors and computational resources which enable them to detect surroundings and enhance driving safety. VEoTC (Vehicular Edge of Things Computing) solutions aim to exploit these embedded sensors and resources to provide computational services to other users. VEoTC can enhance the Quality of Experience (QoE) of vehicle and mobile users requesting computational tasks by providing context-aware services closer to the users that are otherwise not easily accessible in real time. Additionally, such solutions can extend the computational coverage to areas lacking Roadside Unit (RSU) infrastructure. However, VEoTC frameworks face several challenges in effectively localizing and allocating the distributed resources and offloading tasks successfully due to the high mobility of vehicles and fluctuating user densities. The paper proposes a distributed Machine Learning (ML)-based solution which optimizes task scheduling to smart vehicles and/or RSUs through joint resource allocation and task offloading. We formulate a belief-based optimization problem to maximize the QoE of vehicular users while providing performance guarantees that account for geospatial uncertainty associated with the availability of embedded resources. We propose a Deep Reinforcement Learning (DRL)-based solution to solve the formulated problem in real-time adapting to the dynamic network conditions. We analyze the performance of the proposed approach as compared to benchmark optimization and other ML-based techniques. Furthermore, we conduct hardware-based field test experiments to verify the effectiveness of our proposed algorithm to satisfy the stringent real-time latency requirements for various vehicular applications. According to our extensive simulation and experimental results, the proposed solution has the potential to satisfy the stringent QoE guarantees required for critical road safety applications.