Employing deep reinforcement learning for machining process planning: An improved framework

Abstract

Utilizing Deep Reinforcement Learning (DRL) in machining process planning presents a promising avenue to enhance automation, efficiency, and adaptability to diverse scenarios. The definition of the environment plays a crucial role in ensuring the effective application of DRL algorithms, serving as the conduit for formalizing real-world problems into reinforcement learning frameworks. Within the realm of machining process planning, the definition of the environment typically revolves around harnessing components such as processing status, machining operations, and machining resources to reasonably specify the states, actions, reward mechanisms, and other pertinent elements essential for the operation of the DRL algorithm. However, existing DRL-based methods are hampered by various limitations in the definition of the environment. These limitations result in reduced exploration and learning efficiency of the agent, consequently yielding suboptimal machining process planning results. To address these challenges, this paper presents an improved DRL-based framework for machining process planning, specifically targeting aluminum aircraft structural parts. In this context, the framework improves the definition of the state, action, and reward mechanism within the environment, as well as the policy network within the agent. These improvements effectively confine the agent's exploration within a solution space consisting of feasible machining processes for features, thereby mitigating a multitude of invalid explorations and significantly enhancing exploration and learning efficiency. Moreover, these improvements bolster the practical utility of the methodology. In addition, to conduct a more comprehensive exploration for further pursuing optimal solutions, we investigate the incorporation of the Monte Carlo Tree Search algorithm into the proposed framework during the machining process planning phase. Experimental validation conducted on aircraft structural parts demonstrates the efficacy of the proposed method for machining process planning in this domain. Comparative analysis against existing methodologies further underscores the capacity of our framework to generate optimal or near-optimal machining process planning schemes. In conclusion, the proposed framework contributes to advancing machining process planning methods and facilitates wider adoption of DRL within process planning applications.