Comprehensive Decision-Making for Picking and Replenishment in a DQN-Based Hybrid “Parts-To-Picker” Order Picking System

Some large distribution centers have introduced a hybrid “parts-to-picker” order picking system consisting of a pallet warehouse and a tote warehouse to meet diverse order requirements. This system enables collaborative operations between two warehouses during picking and centralized replenishment processes. Additionally, it innovatively allows surplus goods remaining on pallets after picking to be replenished into the tote warehouse. Therefore, making informed decisions during operations such as picking, centralized replenishment, and picking replenishment will significantly enhance overall warehouse operational efficiency. However, to the best of our knowledge, such research has not yet been conducted. This paper addresses the comprehensive decision-making problem of replenishment and picking in a hybrid “parts-to-picker” order picking system. To solve it, we propose an intelligent decision-making framework based on deep reinforcement learning (DQN). We design a state space that incorporates predictive orders, composite warehouse inventory, and warehouse unit status. Furthermore, we design an action space that includes centralized replenishment, picking replenishment, and picking actions. This approach ultimately achieves three objectives: the allocation of quantities for pallet and tote picking, the allocation of quantities for centralized replenishment, and decisions on whether to replenish after picking. The DQN model also combines a reward function that includes penalty factors with an $$-greedy decay strategy, effectively improving the goal of order processing efficiency. The experimental results show that, compared with traditional scheduling strategies and intelligent algorithms, the decision-making model trained by the DQN architecture proposed in this paper can respond quickly in the comprehensive decision-making of picking and replenishment in a hybrid “parts-to-picker” order picking system, and can significantly improve system efficiency. The DQN model improves efficiency by approximately 44% and 17% compared to empirical decision-making and meta-heuristic algorithms, respectively. This study provides a solution that combines theoretical innovation and engineering feasibility for the multi-functional collaborative optimization of smart warehouse logistics systems, demonstrating significant practical value.