MR-SFAMA-Q: A MAC Protocol based on Q-Learning for Underwater Acoustic Sensor Networks

Abstract

In recent years, with the rapid development of science and technology, many new technologies have made people’s exploration of the ocean deeper and deeper, and due to the requirements of national defense and marine development, the underwater acoustic sensor network (UASN) has been paid more and more attention. Nevertheless, the underwater acoustic channel has the properties of considerable propagation delay, limited bandwidth, and unstable network topology. In order to improve the performance of the medium access control (MAC) protocol in UASN, we propose a new MAC protocol based on the Slotted-FAMA of Multiple Reception (MR-SFAMA) protocol. The protocol uses the Q-Learning algorithm to optimize the multi-receiver handshake mechanism. The current state is judged according to the received node request, and the Q-table is established. Through the multi-round interaction between the node and the environment, the Q-table is continuously updated to obtain the optimal strategy and determine the optimal data transmission scheduling scheme. The reward function is set according to the total back-off time and frame error rate, which can reduce the packet loss rate during network data transmission while reducing the delay. In addition, the matching asynchronous operation and uniform random back-off algorithm are used to solve the problem of long channel idle time and low channel utilization. This new protocol can be well applied to unstable network topology. The simulation results show that the protocol performs better than Slotted-FAMA and MR-SFAMA regarding delay and normalized throughput.