Deep Reinforcement Learning for Caching in D2D-Enabled UAV-Relaying Networks

Abstract

Unmanned aerial vehicle (UAV)-relaying can forward files for user devices, but also faces the challenge of the traffic blockage of wireless backhaul. In this paper, we propose a novel caching strategy to pre-cache some popular files at both UAV and user devices to reduce duplicate transmissions in device-to-device (D2D)-enabled UAV-relaying networks. Considering the quality of experience (QoE) of the requesting users, we formulate a file access delay minimization problem by optimizing the cache placement. Due to the dynamics of the environment and the complexity of the formulated problem, we propose a deep deterministic policy gradient (DDPG)-based cache placement optimizing algorithm to decide which files to be cached and where to be cached. In addition, we also analyze theoretically the complexity of the proposed algorithm. Numerical results show our proposed scheme outperforms other baselines.