An intelligent MIMO run-to-run controller for semiconductor manufacturing processes based on an enhanced twin-delayed deep deterministic policy gradient algorithm

Abstract

Achieving accurate target tracking in semiconductor manufacturing processes with complex nonlinearities, strong coupling, and uncertain disturbance environments poses a formidable challenge to run-to-run (RtR) control. In this study, we propose an innovative approach for the online refinement of multi-input multi-output double exponentially weighted moving average (dEWMA) controllers by applying deep reinforcement learning (DRL) techniques. This method harnesses the dynamic interaction capabilities of DRL with the operational environment, facilitating the adaptive tuning of dEWMA parameters to improve the control performance. To further enhance the learning efficiency of the DRL agent, a lightweight DRL model is proposed by combining the structural control network (SCN) with the twin-delayed deep deterministic policy gradient (TD3) algorithm. The SCN component improves the control efficiency by partitioning the policy network into linear and nonlinear modules, enabling the extraction of both local and global features for more effective control. Accordingly, a composite control strategy that synergizes SCN-TD3 with dEWMA is developed. The effectiveness and superiority of the proposed method are rigorously validated through comprehensive comparisons over various disturbance scenarios in both linear and nonlinear chemical mechanical polishing processes. These findings highlight the potential of the proposed DRL-based approach for intelligent RtR control and contribute to yield improvement in semiconductor manufacturing.