Double-layer Q-learning guided NSGA-II for integrated production scheduling and inventory decision considering multi-product orders

High-quality spare parts production and supply are critical for establishing sustained competitive advantages. However, limited manufacturing resources and tight delivery deadlines challenge enterprises in balancing delivery reliability and inventory capital occupation. The widespread existence of multi-product spare parts orders, characterized by interdependent delivery constraints, further complicates production resource allocation and operational coordination. To address these issues, this study treats production and inventory as dual sources for order fulfillment and investigates the integrated production scheduling and inventory decision optimization problem considering multi-product orders (IPSID-MPO), aiming to simultaneously minimize total inventory capital occupation and total order delay penalty. We propose a novel double-layer Q-learning-guided non-dominated sorting genetic algorithm-II (DQ-NSGA), incorporating key innovations: (i) a decoding strategy that considers product delivery constraints, (ii) a hybrid initialization mechanism integrating four problem-specific heuristics, (iii) knowledge-driven local search strategies, and (iv) a self-adaptive adjustment mechanism via double-layer Q-learning. The outer layer dynamically tunes crossover/mutation probabilities based on population evolution, while the inner layer guides individual-specific search strategies. Comprehensive experiments on 180 benchmark instances demonstrate DQ-NSGA’s superiority over mainstream comparative algorithms. Comparisons with common simplified models that consider single-product orders demonstrates the necessity of incorporating multi-product orders in production and inventory decision-making. Furthermore, compared to Make-to-Stock and Make-to-Order paradigms, the proposed IPSID-MPO model not only reduces inventory carrying costs by 28.7% but also effectively enhances the flexibility of the manufacturing system.