A reinforcement learning-driven framework for the Q-commerce multi-product unit scheduling problem

Abstract

With the rapid emergence of Quick commerce (Q-commerce), e-commerce fulfillment systems are shifting from hour-level to minute-level responsiveness. However, this ultra-fast delivery mode introduces complex operational challenges such as multi-warehouse coordination, multi-stage processing, and high-frequency, small-batch order scheduling. To address these challenges, this study formulates and formally defines the Q-commerce multi-product unit scheduling problem with transportation and setup times, which systematically characterizes the outbound scheduling process under multi-resource collaboration. Given the problem’s strong NP-hard nature, a double Q-Learning-based variable neighborhood search (DQL-VNS) algorithm is developed. This algorithm integrates reinforcement learning with metaheuristic optimization to adaptively select neighborhood operators and adjust perturbation intensity, thereby enabling intelligent self-learning within complex search spaces. Extensive computational experiments show that DQL-VNS effectively reduces makespan and total tardiness. In large-scale instances, it achieves over a 10% reduction in average order delay compared with benchmark algorithms. Moreover, the results reveal that the multi-product unit decomposition strategy significantly enhances outbound efficiency and reduces tardiness. In terms of system configuration, the multi-warehouse-few-station mode outperforms the few-warehouse-multi-station mode by achieving better workload balance and greater fulfillment responsiveness. Additionally, due-date flexibility has a substantial impact on scheduling performance, emphasizing its critical role in maintaining customer satisfaction and delivery reliability. Overall, this study presents a novel modeling perspective and an intelligent optimization framework for outbound scheduling and resource coordination in Q-commerce, providing both theoretical and practical insights for developing responsive and sustainable instant retail logistics systems.