Multi-agent deep reinforcement learning based multiple access for underwater cognitive acoustic sensor networks

Abstract

Considering the challenges posed by the significant propagation delays inherent in underwater cognitive acoustic sensor networks, this paper explores the application of multi-agent deep reinforcement learning for the design of multiple access protocols. We deal with the problem of sharing channels and time slots among multiple sensor nodes that adopt different time-slotted MAC protocols. The multiple intelligent nodes can independently learn the strategies for accessing available idle time slots through the proposed multi-agent deep reinforcement learning (DRL) based multiple access control (MDRL-MAC) protocol. Considering the long propagation delay associated with underwater acoustic channels, we reformulate proper state, action, and reward within the DRL framework to address the multiple access challenges and optimize network throughput. To mitigate the decision deviation stemming from partial observability, the gated recurrent unit (GRU) is integrated into DRL to enhance the deep neural network’s performance. Additionally, to ensure both the maximization of network throughput and the maintenance of fairness among multiple agents, an inspiration mechanism (IM) is proposed to inspire the lazy agent to take more actions to improve its contribution to achieve multi-agent fairness. The simulation results show that the proposed protocol facilitates the convergence of network throughput to optimal levels across various system configurations and environmental conditions.