Multi-agent deep reinforcement learning for cross-layer scheduling in mobile ad-hoc networks

Due to the fading characteristics of wireless channels and the burstiness of data traffic, how to deal with congestion in Ad-hoc networks with effective algorithms is still open and challenging. In this paper, we focus on enabling congestion control to minimize network transmission delays through flexible power control. To effectively solve the congestion problem, we propose a distributed cross-layer scheduling algorithm, which is empowered by graph-based multi-agent deep reinforcement learning. The transmit power is adaptively adjusted in real-time by our algorithm based only on local information (i.e., channel state information and queue length) and local communication (i.e., information exchanged with neighbors). Moreover, the training complexity of the algorithm is low due to the regional cooperation based on the graph attention network. In the evaluation, we show that our algorithm can reduce the transmission delay of data flow under severe signal interference and drastically changing channel states, and demonstrate the adaptability and stability in different topologies. The method is general and can be extended to various types of topologies.