Knowledge transfer Q-learning for vessel route planning using automatic identification system-derived expert trajectories

Abstract

Traditional route recommendation systems optimize navigation paths using environmental variables such as weather and sea conditions, but often fail to account for real-world factors encountered by mariners. To address this gap, this study proposes a knowledge transfer Q-learning (KT-QL) algorithm, a reinforcement learning method built upon the Q-learning framework. The proposed KT-QL algorithm integrates expert trajectory knowledge derived from Automatic Identification System data into the Q-learning process, enabling the agent to combine trial-and-error exploration with data-driven guidance. Experimental results show that KT-QL reduces Hausdorff distances by approximately 39 % compared with conventional reinforcement learning and traditional search methods, and enhances fuel consumption prediction accuracy by approximately 2 %. These findings highlight the potential of KT-QL to enhance maritime operational efficiency, safety, and environmental sustainability.