Narrowband Active Noise Control with DDPG Based on Reinforcement Learning

Abstract

This study investigates the use of deep reinforcement learning for active noise control (DRL-ANC) to cancel narrowband noise. The filtered-x least mean square algorithm for ANC, which includes the secondary path model in itself, has been widely used in various applications. If the path model is inaccurate due to the variations of the actual path, control performance and stability of the algorithm can be restricted. To eliminate the effect by the model inaccuracy, it is considered to remove the path model in the novel DRL-ANC strategy. A DRL approach using the deep deterministic policy gradient without any path model is adopted to learn the behavior of a physical environment including the effect of the actual secondary path in real time. However, a temporal credit assignment problem arises due to the time-delayed reward inherent in the secondary path, which means that the current action could not be evaluated by its true. To address this problem, this study proposes a novel definitions of the state and action of the RL agent, specialized in narrowband noise suppression. Additionally, a novel exploration noise is also suggested to enhance effectiveness and practicality of the learning process. Computer simulations and real-time control experiments were conducted, and the results demonstrated that the proposed DRL-ANC algorithm can robustly cope with changes in the secondary path.