A Novel Reinforcement Learning-based Unsupervised Fault Detection for Industrial Manufacturing Systems

Abstract

With the advent of industry 4.0, machine learning (ML) methods have mainly been applied to design condition-based maintenance strategies to improve the detection of failure precursors and forecast degradation. However, in real-world scenarios, relevant features unraveling the actual machine conditions are often unknown, posing new challenges in addressing fault diagnosis problems. Moreover, ML approaches generally need ad-hoc feature extractions, involving the development of customized models for each case study. Finally, the early substitution of key mechanical components to avoid costly breakdowns challenge the collection of sizable significant data sets to train fault detection (FD) systems. To address these issues, this paper proposes a new unsupervised FD method based on double deep-Q network (DDQN) with prioritized experience replay (PER). We validate the effectiveness of the proposed algorithm on real steel plant data. Lastly, we compare the performance of our method with other FD methods showing its viability.