Deep reinforcement learning for the real-time inventory rack storage assignment and replenishment problem

Abstract

The e-commerce industry is quickly transforming towards more automation and technological advancements. With the growing intricacy of warehouse operations, there is a need for control systems that can efficiently handle this complexity. This study considers a Robotic Mobile Fulfillment System (RMFS), a semi-automated warehousing system. This system employs autonomous mobile robots (AMRs) to retrieve inventory racks from the storage area; this way, human activity is eliminated within the storage area itself. The fleet of robots both store and retrieve the inventory racks to either workstations, where human pickers are stationed that pick items from the racks, or replenishment stations, where depleted inventory racks can be restocked with items. An attractive characteristic of the RMFS is that it dynamically changes the positioning of the inventory racks based on the frequency of inventory rack requests and the state of their stock levels. The optimization objective considered in this study for the dynamic positioning problem of the racks within the storage area is to minimize the average cycle time of the mobile robots to perform retrieval and replenishment activities. We propose a deep reinforcement learning approach to train a decision-making agent to learn a policy for the storage assignment and replenishment of inventory racks. The learned policy is compared to the commonly used decision rules in the academic literature on this problem. The experimental results show the potential benefits of training an agent to learn a storage and replenishment policy. Cycle time improvements up to 5.4 % can be achieved over the best-performing decision rules. This research contributes to advancing the understanding of intelligent storage assignment and replenishment strategies for the real-time decision-making process within an RMFS.