Manufacturing resource-based self-organizing scheduling using multi-agent system and deep reinforcement learning

Abstract

Enterprises are vigorously developing smart factories to meet the approaching mass customization. As a promising control paradigm for smart factories, the self-organizing scheduling mode can build networked manufacturing things. Compared to the global control of traditional scheduling methods, its decentralized control can provide stronger dynamic response and self-regulation capabilities. Therefore, this paper proposes a self-organizing scheduling method based on multi-agent system (MAS) and deep reinforcement learning (DRL) for smart factory. Firstly, a novel MAS with partially decentralized control architecture is established, where the manufacturing resources and cloud are constructed as agents. Then, unlike traditional methods, a self-organizing negotiation mechanism based on contract network protocol is designed for production-logistics collaboration. Considering problem domain knowledge, logistics task bidding of automated guided vehicle agents is based on heuristics, and processing task bidding of machine agents is based on multi-agent DRL. It can ensure the timely delivery of orders, rapid logistics process and efficient production. Finally, machine agents embedded with DRL adopt the centralized training and decentralized execution framework. An action space based on three priorities is designed to ensure the correct bidding of each machine agent and reasonable auction of processing tasks. Experimental results show that compared with scheduling rules, genetic programming and three DRL methods, the proposed method achieves better scheduling performance through reasonable competition of heterogeneous resource agents, and can effectively handle new job arrivals and machine breakdowns.