Adaptive hybrid priority-enabled deep Q-network for machine degradation-based dynamic scheduling of assembly line

Abstract

As production systems evolve, the performance degradation of machines can result in increased downtime and lower throughput; preventive maintenance is needed to fully or partially restore the initial performance of machines, making dynamic scheduling essential. This problem is modeled in this study as a Markov decision process, where each machine agent assigns components and their placement sequences based on the dynamic production status of the assembly line. This approach enables intelligent adaptation to changing machine conditions, enhances efficiency, and minimizes delays. An intelligent decision-making framework has been developed to handle agile planning and the scheduling of dynamic problems of an assembly line. This multi-layer framework is integrated with a virtual simulation model of the existing physical production system, enabling informed decision-making. An adaptive hybrid priority deep Q-network (AHP-DQN) algorithm is proposed, utilizing a distributed multi-agent system for decentralized decision-making among machines on the assembly line. To enhance the performance of the proposed AHP-DQN algorithm in environments characterized by complex state representations and uncertain machine performance degradation, two key components are customized: the neural network and experience replay buffer mechanism. First, a partially connected neural network incorporating noisy layers is employed to enhance exploration efficiency and robustness. Second, an adaptive hybrid prioritized experience replay buffer is introduced by combining line balancing with the rate to balance sampling quality and efficiency. The deep Q-network, enhanced with neural networks and experience priority mechanisms, enables the system to learn optimal balancing policies through interaction with its environment. Simulation results demonstrate that the adaptive hybrid model outperforms traditional balancing algorithms in terms of throughput and overall system performance. After investigating the performance of the AHP-DQN, it is validated that the proposed algorithm outperforms the existing approaches. Specifically, compared to the commonly used earliest completion time rule in industrial applications, the proposed method achieves a performance improvement of approximately 4–10%, highlighting its practical applicability and effectiveness.