Reinforcement-Learning-Based AAV 3-D Target Tracking and Digital-Twin-Assisted Collision Avoidance With Integrated Sensing and Communication

The flexibility and maneuverability of autonomous aerial vehicles (AAVs) lend themselves to tracking users and operating as an aerial base station carrying out communication enhancement functionality. A core challenge neglected by most existing works is that the true-but-unknown obstacles can jeopardize AAV flight security and shadow its communication links with users, resulting in poor achievable rate and high collision risks. In this article, a deep-reinforcement-learning (DRL)-based AAV target tracking and digital-twin (DT)-assisted collision avoidance method is proposed to optimize AAV’s communication performance while tracking moving users. Toward this end, twin delayed deep deterministic policy gradient (TD3) as a novel and policy-based DRL algorithm is used to construct an agent responsible for adaptive deciding AAV flying control actions. To efficiently detect unknown obstacles in a flight environment, an orthogonal frequency-division multiple (OFDM)-based integrated sensing and communication (ISAC) system is investigated, endowing AAV’s agent with real-time obstacle distance. Finally, we present a DT obstacle model construction mechanism and integrate it with TD3 agent training. The extensive simulations demonstrate the reward convergence of the TD3 algorithm and the communication improvement with stable user tracking and reliable collision avoidance, compared with conventional approaches.