A knowledge-based matheuristic driven by dueling Q-learning for integrated task planning and lot-streaming scheduling in cloud manufacturing

Abstract

As a new service-oriented paradigm, cloud manufacturing (CMfg) virtualizes distributed manufacturing resources into services orchestrated via cloud platforms to fulfill individualized production demands. With the increasing need for mass customization and ever-shortening production cycles, higher flexibility and efficiency in CMfg resource assignment and task scheduling become particularly essential. Within this context, the integrated optimization of CMfg tasks planning and scheduling is receiving increased attention. Among the existing studies on this problem, however, it is usually assumed that either a single fixed routing or a predefined set of alternative process routes is given in advance, overlooking the possibility of gaining better solutions by optimizing the process routes in task planning with consideration of the resource heterogeneity and availability. Another common assumption is that the task is processed as a whole, which may result in lower scheduling flexibility. To address these issues, this paper integrates routing optimization with lot streaming scheduling at the sublot level, instead of task level, and incorporates two flexibility-enhancing mechanisms: variable sublots and intermingling. In dealing with the high solving complexity, a knowledge-based matheuristic driven by dueling Q-learning is proposed, in which eight MILP-based optimization strategies are designed, each targeting a core subproblem. A knowledge-based pruning strategy is elaborated for targeted improvement, while the dueling Q-learning mechanism adaptively guides the selection of MILP-based strategies. The effectiveness and efficiency of the proposed approach are verified through extensive experiments based on the Kim benchmark, with its practical applicability and advantages being demonstrated through a real-world industrial case.