Multi-objective task offloading for highly dynamic heterogeneous Vehicular Edge Computing: An efficient reinforcement learning approach

Abstract

Vehicular Edge Computing (VEC) provides a flexible distributed computing paradigm for offloading computations to the vehicular network, which can effectively solve the problem of limited vehicle computing resources and meet the on-vehicle computing requests of users. However, the conflict of interest between vehicle users and service providers leads to the need to consider a variety of conflict optimization goals for computing offloading, and the dynamic nature of vehicle networks, such as vehicle mobility and time-varying network conditions, make the offloading effectiveness of vehicle computing requests and the adaptability to complex VEC scenarios challenging. To address these challenges, this paper proposes a multi-objective optimization model suitable for computational offloading of dynamic heterogeneous VEC networks. By formulating the dynamic multi-objective computational offloading problem as a multi-objective Markov Decision Process (MOMDP), this paper designs a novel multi-objective reinforcement learning algorithm EMOTO, which aims to minimize the average task execution delay and average vehicle energy consumption, and maximize the revenue of service providers. In this paper, a preference priority sampling module is proposed, and a model-augmented environment estimator is introduced to learn the environmental model for multi-objective optimization, so as to solve the problem that the agent is difficult to learn steadily caused by the highly dynamic change of VEC environment, thus to effectively realize the joint optimization of multiple objectives and improve the decision-making accuracy and efficiency of the algorithm. Experiments show that EMOTO has superior performance on multiple optimization objectives compared with advanced multi-objective reinforcement learning algorithms. In addition, the algorithm shows robustness when applied to different environmental settings and better adapting to highly dynamic environments, and balancing the conflict of interest between vehicle users and service providers.