A distributed multi-agent joint optimization algorithm based on CERL and A2C for resource allocation in vehicular networks

Abstract

Vehicular networking plays an indispensable role in enhancing road safety and traffic efficiency. Although existing technologies have made significant progress in reusing vehicle-to-infrastructure (V2I) link resources for vehicle-to-vehicle (V2V) links, they still face challenges such as the high dimensionality of the joint action space and unsatisfactory optimization with limited in-vehicle radio resources, variable environments, and uncertainties. Reinforcement learning-based joint algorithms that separately optimize channel allocation and power selection can reduce the dimensionality of the joint action space. However, it is difficult to effectively coordinate channel allocation and power selection, which greatly affects the performance of them. To address these challenges, this paper proposes a distributed multi-agent joint optimization algorithm based on a novel cross-entropy loss-based reinforcement learning (CERL) algorithm and the A2C algorithm for separately optimizing channels and power in vehicular networks. Furthermore, a multi-round stochastic search strategy is presented to optimize the experience pools and coordinate the channel allocation and the power selection for the proposed distributed multi-agent joint optimization algorithm. With the help of the multi-round stochastic search strategy, the proposed distributed multi-agent joint optimization algorithm can significantly improve the optimization performance in resource allocation. To evaluate the performance of the proposed distributed multi-agent joint optimization algorithm in both the V2V link transmission success rate and the V2I link throughput, a comprehensive simulation study is conducted under different channel resource availability scenarios with different sizes of security data. The experimental results demonstrate that our proposed algorithm can significantly improve the V2I link throughput and the V2V link transmission success rate, and outperforms the existing algorithms in terms of radio efficiency. Specifically, under two different channel resource availability scenarios, our proposed algorithm can achieve more than 99.9 % average V2V link transmission success rate and 2.99 Mbps and 2.07 Mbps higher average V2I link throughput than the competitive algorithm D3QN-LS when the security data size ranges from 1 × 1060 Bytes to 8 × 1060 Bytes. The proposed algorithm theoretically provides a new perspective and solution for separately optimizing channels and power in high-dimensional complex dynamic environments of vehicular networks.