UAV-assisted Internet of vehicles: A framework empowered by reinforcement learning and Blockchain

Abstract

This paper addresses the challenges of selecting relay nodes and coordinating among them in UAV-assisted Internet-of-Vehicles (IoV). Recently, UAVs have gained popularity as relay nodes to complement vehicles in IoV networks due to their ability to extend coverage through unbounded movement and superior communication capabilities. The selection of UAV relay nodes in IoV employs mechanisms executed either at centralized servers or decentralized nodes, which have two main limitations: 1) the traceability of the selection mechanism execution and 2) the coordination among the selected UAVs, which is currently offered in a centralized manner and is not coupled with the relay selection. Existing UAV coordination methods often rely on optimization methods, which are not adaptable to different environment complexities, or on centralized deep reinforcement learning, which lacks scalability in multi-UAV settings. Overall, there is a need for a comprehensive framework where relay selection and coordination processes are coupled and executed in a transparent and trusted manner. This work proposes a framework empowered by reinforcement learning and Blockchain for UAV-assisted IoV networks. It consists of three main components: a two-sided UAV relay selection mechanism for UAV-assisted IoV, a decentralized Multi-Agent Deep Reinforcement Learning (MDRL) model for efficient and autonomous UAV coordination, and finally, a Blockchain implementation for transparency and traceability in the interactions between vehicles and UAVs. The relay selection considers the two-sided preferences of vehicles and UAVs based on the Quality-of-UAV (QoU) and the Quality-of-Vehicle (QoV). Upon selection of relay UAVs, the coordination between the selected UAVs is enabled through an MDRL model trained to control their mobility and maintain the network coverage and connectivity using Proximal Policy Optimization (PPO). MDRL offers decentralized control and intelligent decision-making for the UAVs to maintain coverage and connectivity over the assigned vehicles. The evaluation results demonstrate that the proposed selection mechanism improves the stability of the selected relays, while MDRL maximizes the coverage and connectivity achieved by the UAVs. Both methods show superior performance compared to several benchmarks.