Dynamic attention for container relocation problems: a deep reinforcement learning approach

Abstract

The Container Relocation Problem (CRP) is a key challenge in terminal operations. It aims to identify an optimal sequence for moving containers, minimizing the total number of container rehandling operations within a container yard bay. The Dynamic Container Relocation Problem (DCRP) further complicates this task this by incorporating the dynamic arrival of containers, necessitating rapid rescheduling of retrieval sequences. In this paper, we present a deep reinforcement learning framework to address both CRP and DCRP under a unified policy network. To handle dynamic layout changes, we introduce a novel dynamic attention mechanism that integrates long short-term memory and multi-head attention layers, enabling the model to adaptively capture spatial and temporal dependencies in container stacks. Trained with policy gradient optimization, our approach learns to balance immediate placement decisions with long-term retrieval efficiency without relying on domain-specific heuristics. The experiments demonstrate that our approach outperforms traditional methods for both CRP and DCRP, highlighting the potential of deep reinforcement learning in solving CRPs without extensive expert knowledge.