Innovative edge caching: A multi-agent deep reinforcement learning approach for cooperative replacement strategies

Abstract

Cooperative edge caching has emerged as a promising solution to alleviate the traffic burden of backhaul and improve the Quality of Service of 5G applications in the 5G era. Several cooperative edge caching methods alleviate the traffic burden of backhaul by transmitting contents between edge nodes cooperatively, thereby reducing the delay of content transmission. However, these methods have a limited ability to handle complex information in multi-edge scenarios, and they mainly focus on cooperation in content transmission while scarcely considering cooperation in cache replacement. As a result, they cannot effectively utilize the cache space of collaborative edges, leading to suboptimal system utility. In this paper, we propose a cache replacement strategy for cooperative edge caching based on a novel multi-agent deep reinforcement learning network. Firstly, we present a cooperative edge caching model aimed at maximizing the system throughput. Then, we formulate the cache replacement process in the cooperative edge caching system as a Markov Game (MG) model. Finally, we design a Discrete MADDPG algorithm based on a discrete multi-agent actor-critic network to derive the cache replacement strategy and effectively manage content redundancy within the system. Simulation results demonstrate that our proposed algorithm achieves higher system throughput while effectively controlling content redundancy.