A five-dimensional digital twin framework driven by large language models-enhanced RL for CNC systems

In response to the growing demand for intelligence and digitalization in computer numerical control (CNC) systems, particularly in virtual debugging, performance evaluation, and machining quality optimization within the manufacturing sector, this study explores the mapping process from physical entities in the physical space to twin entities in the digital twin space. It further delves into the data transmission layer that represents the physical mechanisms of these entities. By constructing multi-layered models, digital twin technology facilitates the virtual simulation and personalized services of multi-layer coupled units in CNC systems. However, given the complexity and diversity of internal units in CNC systems, traditional CNC machining systems remain overly reliant on human expertise, while existing digital twin frameworks lack effective representation and efficient optimization methods tailored to the CNC field. This paper proposes a novel five-dimensional framework for digital twins in CNC systems to address these challenges, driven by large language models-assisted enhanced reinforcement learning. The framework leverages the advantages of digital twin technology in dynamic process management and reinforcement learning in intelligent analysis and decision-making. It comprises five key layers: the physical entity layer, the virtual entity layer, the intelligent decision-making layer, the data transmission layer, and the real-time computation layer. It also encompasses multi-domain modeling, including information models, mechanism models, and digital threads, and explores how large language models (LLM) can assist and enhance reinforcement learning in CNC applications. Finally, the study applies the framework to the tool-axis vector planning problem in CNC systems, positioning the LLM as an indirect decision-maker within reinforcement learning. Experimental results demonstrate that integrating LLM-enhanced reinforcement learning algorithms within the CNC system’s digital twin framework effectively reduces tool-axis vector variation rates, thereby decreasing collision interference incidents during machining. To some extent, the proposed digital twin framework’s effectiveness has been validated, providing a valuable approach for the advancement of the CNC system field.