Real-time response to machine failures in self-organizing production execution using multi-agent reinforcement learning with effective samples

With the growing demand for personalized production, multi-agent technology has been introduced to facilitate rapid self-organizing production execution. The application of communication protocols and dynamic scheduling algorithms supports multi-agent negotiation and real-time scheduling decisions in response to conventional production events. To address machine failures, real-time response strategies have been developed to manage jobs affected by the disruptions. However, the performance of existing strategies varies significantly depending on the real-time production state. In this paper, we propose a real-time response strategy using multi-agent reinforcement learning (MARL) that provides an appropriate response strategy for each job affected by machine failures, considering the real-time production state. Specifically, we establish a self-organizing production execution process with machine failures to specify the real-time response problem. Subsequently, a Markov game involving multiple buffer agents is constructed, transforming the real-time response problem into a MARL task. Furthermore, a continuous variable ranging from 0 to 1 is defined as the action space for each buffer agent, allowing it to select a response strategy for each affected job. Finally, a modified multi-agent deep deterministic policy gradient (MADDPG) algorithm is introduced, leveraging effective samples to train buffer agents at each failure moment. This enables the selection of an optimal response strategy for each affected job. Experimental results indicate that the proposed real-time response strategy outperforms both existing response strategies and the original MADDPG-based strategy across 54 distinct production configurations.