A multi-agent reinforcement learning based scheduling strategy for flexible job shops under machine breakdowns

Abstract

In a highly disrupted workshop environment, machine failures may occur frequently, requiring real-time schedule repair strategies. This paper proposes a type-aware multi-agent deep reinforcement learning (MADRL) to address real-time schedule repair for the flexible job shop scheduling problem under machine breakdowns. First, the problem is modeled as a multi-agent Markov decision process. At each decision point, the relationships among machine agents and operations are represented using a heterogeneous graph. Based on the graph, machine node embeddings are obtained based on a meta-path type-aware (MPTA) recurrent neural network. Meanwhile, a heterogeneous graph attention network is introduced to aggregate the features of nodes in the heterogeneous graph, forming operation embeddings available for each machine agent’s selection. The type-aware process of obtaining machine node embeddings and operation embeddings utilizes a hypernetwork to achieve parameter adaptation of node types, edge types, and node locations. Finally, performing a cross-attention mechanism on machine node embedding and its candidate operation embeddings for selection. Compared with the heuristic rules and MADRL algorithms, numerical experiment results indicate that the proposed MADRL achieves the minimum stability objective while reducing the makespan of the schedule affected by machine breakdowns.