PDRL-CM: An efficient cooperative caching management method for vehicular networks based on deep reinforcement learning

In vehicular networks, onboard devices face the challenge of limited storage, and computational resources constrain their processing and storage capabilities. This limitation is particularly significant for applications that require complex computations and real-time responses. Additionally, limited storage capacity reduces the range of cacheable data, which can impact the immediate availability of data and the continuity of services. Therefore, improving cache utilization and meeting vehicles’ real-time data demands pose significant challenges. Deep reinforcement learning can optimize the issues arising from agents’ continuously changing state and action spaces due to increasing request demands. However, training the network may encounter instability and convergence difficulties in dynamic and complex environments or situations with sparse rewards. In response to these issues, this paper proposes a Priority-based Deep Reinforcement Learning Collaborative Cache Management method (PDRL-CM). PDRL-CM first designs a lightweight cache admission strategy that leverages data’s inherent and combined attributes. It then makes cache admission decisions Using Monte Carlo sampling and a max-value search strategy combined with a feedforward neural network. Secondly, the method considers minimizing system latency and reducing vehicle energy consumption as joint optimization problems. An improved deep reinforcement learning algorithm solves this problem and makes cache-sharding decisions. A prioritized experience replay mechanism is incorporated to adjust the network prediction model quickly and accelerate the convergence process. Experimental results indicate that, compared to existing DRL-based caching methods, PDRL-CM offers faster data processing efficiency and higher cache hit rates under varying vehicle density, storage capacity, and content volume conditions.