QoE-Driven Proactive Caching With DRL in Sustainable Cloud-to-Edge Continuum

Abstract

Cloud-assisted edge computing scenarios can intelligently cache and update the content periodically, thereby enhancing users’ overall perception of service, which is called quality of experience (QoE). To maximize QoE in cloud-to-edge continuum, we formulate a multi-objective optimization problem, which optimizes the cache hit ratio while simultaneously minimizing traffic load and time latency. Particularly, we present an innovative algorithm named Hyperdimensional Transformer with Priority Experience Playback-based Agent Deep network (HT-PAD), which provides a complete solution for prediction and decision-making for proactive caching. First, to improve the prediction accuracy of cached content, we use the encoding layer in hyperdimensional (HD) computing to extract the information features. Second, HD-Transformer, as the prediction part of HT-PAD, is proposed to make predictions based on user preferences, historical information, and popular information. HD-Transformer uses deep neural networks to predict user preferences and process time series data by combining hyperdimensional computation with the Transformer. Third, to avoid errors in the prediction content, we employ PER-MADDPG as the decision-making part of HT-PAD, which consists of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) and Prioritized Experience Replay (PER). We use MADDPG to improve content decision-making and utilize PER to select appropriate training samples for PER-MADDPG. Finally, our experiments show that our proposed approach achieves strong performance in terms of edge hit ratio, latency, and traffic load, thus improving QoE.