Continual multi-target domain adaptation for industrial process fault diagnosis

Abstract

In multi-operating condition production processes, process data typically arrive continuously with distinct distribution. Domain adaptation techniques are commonly employed to settle the domain shift caused by variations in operating conditions. However, those models trained on continual data streams face the dilemma of adapting to new data while forgetting old knowledge. In this study, a novel transfer learning model called continual multi-target domain adaptation with dual knowledge distillation (CMTDA-DKD) is proposed for process fault diagnosis, which is trained on multiple target domains collected sequentially from varying working conditions. To adapt to the target streams from different working conditions, maximum mean discrepancy and adversarial training are utilized to narrow the distribution gap and guide the feature generator to learn domain invariant features between source and target domains. In addition, a dual knowledge distillation module is proposed to mitigate catastrophic forgetting of previous target domains in both feature and class levels. Moreover, a knowledge bank based on a sample selection module is proposed to restore the representative target domain samples in previous incremental stages, which enables the model to preserve prior knowledge. The application performance of CMTDA-DKD in continuous stirred tank reactor process, three-phase process and a hydraulic system demonstrates its effectiveness and superiority over other methods.