A novel multi-attention reinforcement learning for the scheduling of unmanned shipment vessels (USV) in automated container terminals

Abstract

To improve the operating efficiency of container terminals, we investigate a closed-loop scheduling method in an autonomous inter-terminal system that employs unmanned shipment vessels (USVs) to transport containers among operational berths (Dedicated to USVs) in seaport terminals. Our USVs scheduling model is developed by considering energy replenishment, time windows, and berth restrictions, aiming to obtain cost-saving USV transportation solutions and conflict-free paths. To solve this optimization model more efficiently, we propose the multi-attention reinforcement learning (MARL) algorithm by integrating an encoder-decoder framework and an unsupervised auxiliary network. The MARL algorithm provides instant problem-solving capabilities and benefits from extensive offline training. Experimental results demonstrate that our method can obtain efficient solutions for our USVs scheduling problem, and our algorithm outperforms other compared algorithms on computing time and solution accuracy.