Multi-UAV-enabled AoI-aware WPCN: A Multi-agent Reinforcement Learning Strategy

Abstract

Unmanned Aerial Vehicles (UAVs) have been deployed in virtually all tasks of enabling wireless powered communication networks (WPCNs). To ensure sustainable energy support and timely coverage of terrestrial Internet of Things (IoT) devices, a UAV needs to continuously hover and transmit wireless energy signals to charge these devices in the downlink. Then, the devices send their independent information to the UAV in the uplink. However, it was noted that the majority of existing schemes related to UAV-enabled WPCN are mainly based on a single UAV and cannot meet the requirements of a large-scale WPCN. In this paper, we design a separated UAV-assisted WPCN system, where two UAVs are deployed to behave as a UAV data collector (UAV-DC) and UAV energy transmitter (UAV-ET), respectively. Thus, the collection of fresh information and energy transfer are treated separately at the level of the two corresponding UAVs. These two tasks could be enhanced by optimizing the UAVs’ trajectories. For this purpose, we leverage a multi-agent deep Q-network (MADQN) strategy to provide appropriate UAVs’ trajectories that jointly minimize the expected age of information (AoI), enhance the energy transfer to devices, and minimize the energy consumption of UAVs. Simulation results show that our system enhances the performance of our strategy significantly in terms of AoI and energy transfer compared with baseline methods.