Integrated optimisation of multi-pass cutting parameters and tool path with hierarchical reinforcement learning towards green manufacturing

Five-axis machining, especially flank milling, is popular in machining thin-walled freeform surface parts with high energy consumption. Reducing the machining energy consumption is paramount for advancing green manufacturing. Therefore, this paper proposes an energy-efficient integration optimisation of cutting parameters and tool path with hierarchical reinforcement learning (HRL). Firstly, a novel multi-pass machining energy consumption model is developed with cutting and path parameters, based on which the integrated optimisation problem is modelled considering a dynamic workpiece deformation constraint. Secondly, HRL with a Soft Actor Critic agent (HSAC) decouples the model into two Markov Decision Processes at different timescales. The higher-layer plans cutting parameters for each pass on a macro timescale, while the micro-timescale lower-layer performs multiple tool path expansions with the planned cutting parameters, and provides feedback to the higher layer. By hierarchical optimisation and non-hierarchical interaction, the model is efficiently solved. Moreover, curriculum transfer learning is applied to expedite task completion of the lower layer, enhancing interaction efficiency between the two layers. Experiments show that, compared with two benchmarks, the proposed method improves machining energy consumption by 35.02 % and 30.92 %, and reduces machining time by 38.57 % and 27.17 %, providing a promising paradigm of green practices for thin-walled freeform parts and the broader manufacturing industry.