Curriculum-Based Soft Actor-Critic for Multi-Section R2R Tension Control

Abstract

Precise tension control in roll-to-roll (R2R) manufacturing systems remains challenging under varying operating conditions and process uncertainties. While they perform well for systems with accurate models and small disturbances, model-based controllers face limitations when operating conditions deviate from design points or when process uncertainties are large. This paper presents a curriculum-based Soft Actor-Critic (SAC) approach as an alternative to R2R control under uncertainty. The method learns control policies through progressive exposure to various operating conditions without requiring system-specific parameter tuning. We train SAC with a three-phase curriculum with gradual expansion of reference trajectory diversity, from a narrow ranges (27N–33N) to full operational envelope (20N–40N), enabling the policy to generalize across the complete operational range. Evaluation on a three-section R2R system shows that the learned policy maintains tight tension tracking under nominal conditions and handles large disturbances (20N$\to$40N step changes) well with a single control policy. The proposed approach offers an alternative to traditional model-based methods for applications where operating conditions vary and process uncertainties are significant, typical for R2R manufacturing of flexible devices.