Cross-Regional Task Offloading with Multi-Agent Reinforcement Learning for Hierarchical Vehicular Fog Computing

Vehicular fog computing (VFC) can make full use of computing resources of idle vehicles to increase computing capability. However, most current VFC architectures only focus on the local region and ignore the spatio-temporal distribution of computing resources, resulting that some regions have idle computing resources while others cannot satisfy the requirements of tasks. Therefore, we propose a hierarchical VFC architecture, where neighboring regions can share their idle computing resources. Considering that the existing centralized offloading mode is not scalable enough and the high complexity of cooperative task offloading, we put forward a distributed task offloading strategy based on multi-agent reinforcement learning. Moreover, to tackle the inefficiency caused by the multi-agent credit assignment problem, we provide the counterfactual multi-agent reinforcement learning approach which exploits a counterfactual baseline to evaluate the action of each agent. Simulation results validate that the hierarchical architecture and the distributed algorithm improves the efficiency of global performance.