A Q-learning-based trust model in underwater acoustic sensor networks (UASNs)

Abstract

Underwater acoustic sensor networks (UASNs) play a pivotal role in various civil and military fields. However, due to their open nature, they are susceptible to multiple security threats. As such, developing robust and reliable security strategies is essential to ensure the normal operation of UASNs. This paper proposes a Q-learning-based trust model (QLTM) for UASNs. To detect hostile nodes, each underwater sensor node is required to collect trust evidence –namely energy trust evidence, data trust evidence, and communication trust evidence–through communication and interaction with its neighboring nodes. After gathering the trust evidence, QLTM presents a distributed Q-learning-based trust management model that adapts to dynamic underwater environments. It continuously updates the trust parameters based on ongoing interactions between the agent and the environment. The Q-learning-based trust management model includes a state set with three states: trust, distrust, and uncertain. Additionally, the reward function is calculated according to the gathered trust evidence, and the weight of each trust evidence is determined such that evidence with a lower value carries more weight, thus having a greater effect on the generated reward. Experimental results demonstrate the effectiveness of QLTM compared to other trust mechanisms, so that QLTM improves the detection accuracy rate by 5.04%. However, when the attack mode changes in the network, QLTM performs approximately 4.29% worse than TUMRL in detecting malicious nodes. On the other hand, QLTM reduces the false alarm rate by about 7.39% and increases energy efficiency by approximately 4.26%.