Online sequential decision making of multi-stage assembly process parameters based on deep reinforcement learning and its application in diesel engine production

Abstract

Maintaining fixed parameters during batch assembly of complex mechanical products often results in quality inconsistencies due to time-varying operational conditions, including equipment performance degradation, production environment disturbance, and operator skill variations. This operational reality necessitates online parameter adaptation mechanisms to counteract progressive quality deviations. While complex assemblies inherently involve sequential multi-stage workflows across distributed stations, conventional optimization strategies often employ monolithic parameter adjustments that neglect error propagation effects and inter-stage quality interdependencies. To address the dual challenges of dynamic operating conditions and multi-stage coordination, this study proposes an online sequential decision-making framework based on deep reinforcement learning. First, a causal inference model for assembly quality prognosis is constructed by integrating the greedy equivalence search algorithm with domain-specific expert knowledge, enabling systematic modeling of multi-stage quality dependencies. Subsequently, the multi-stage parameters optimization problem is formalized as a Markov decision process, with innovatively defined state space as assembly progress, action space as adjusted parameters range, and physics-informed reward function derived from quality inference results. Building on this, the proximal policy optimization algorithm is improved by stage-aware experience replay and gradient alignment constraints to learn the optimal policy, and then select the optimal action. Experiments on a real-world diesel engine assembly dataset demonstrate a 17.16 % improvement in product qualification probability, significantly outperforming conventional methods. The proposed framework effectively captures time-varying assembly characteristics and achieves cross-stage parameter coordination through sequential decision-making, offering a novel data-driven solution for quality control in complex product assembly systems.