Joint Physical and Network Layers Design for STARS-Assisted Multi-Cellular Edge Caching

A simultaneously transmitting and reflecting surface (STARS) assisted multi-user downlink multiple-input signal-output (MISO) multi-cellular edge caching system is investigated. The deployment of STARS enhances the coverage of base stations (BSs), particularly at cellular boundaries. However, this advancement introduces a complex user association issue that necessitates the consideration of both caching state and channel state information (CSI). In this paper, we formulate a joint optimization problem involving content caching, user association, active beamforming at BS, and passive beamforming at STARS for minimizing long-term power consumption. We propose two algorithms for the formulated problem: 1) A two time-scale cooperative twin delayed deep deterministic policy gradients (TD3). Considering the distinct time scales of the pushing and delivering phases in edge caching, the Markov decision process (MDP) models of dual time scales are constructed and two deep reinforcement learning (DRL) agents work together to jointly address the optimization problem. 2) A bio-inspired DRL framework, especially, a particle swarm optimization (PSO)-inspired TD3 algorithm is introduced in detail. Inspired by the behavior of the biological population in nature, this algorithm regards agents as individuals and enables the concurrent training of multiple agents while they interact with global information via a biological population information interaction mode, thereby enhancing the performance of power optimization. The numerical results demonstrate that the STARS-assisted multi-cellular edge caching system has advantages over traditional cellular systems, especially in scenarios where the number of mobile users and Zipf skewness factor is large. Moreover, the proposed two time-scale cooperative TD3 and PSO-inspired TD3 algorithms are superior in reducing network power consumption than conventional TD3.