Developing Decision-Making Algorithm for Unmanned Vessel Navigation Using Markov Processes

Abstract

: In this study, the autonomous decision-making architecture of unmanned vessel navigation has been formulated. The aim of this study is the advancement of mathematical methods in the ship transportation field with relevance to collision avoidance scenario applications. The process of seafarers safely navigating a vessel at sea entails enacting appropriate decision-making at the appropriate time. In our model, we do not input the appropriate action order based on a seafarer’s experience. The model scores each step’s reward by its action behaviour and learns how to avoid obstacles by itself. By deploying decision timing, state, reward, and digitizing the seafarer’s decision, we establish a reinforcement learning algorithm based on Markov decision processes. In the model training, under a single factor influence, the vessel tends to change course with the best appropriate action behaviour, which is almost consistent with decision-making behaviour based on actual experience at sea.