Age of Information Minimization in Multicarrier-Based Wireless Powered Sensor Networks.

This study investigates the challenge of ensuring timely information delivery in wireless powered sensor networks (WPSNs), where multiple sensors forward status-update packets to a base station (BS). Time is partitioned to multiple time blocks, with each time block dedicated to either data packet transmission or energy transfer. Our objective is to minimize the long-term average weighted sum of the Age of Information (WAoI) for physical processes monitored by sensors. We formulate this optimization problem as a multi-stage stochastic optimization program. To tackle this intricate problem, we propose a novel approach that leverages Lyapunov optimization to transform the complex original problem into a sequence of per-time-bock deterministic problems. These deterministic problems are then solved using model-free deep reinforcement learning (DRL). Simulation results demonstrate that our proposed algorithm achieves significantly lower WAoI compared to the DQN, AoI-based greedy, and energy-based greedy algorithms. Furthermore, our method effectively mitigates the issue of excessive instantaneous AoI experienced by individual sensors compared to the DQN.