Unlearning Recently Learned Data to Preserve Historical Learning for Dynamic Data Stream Classification

Abstract

At present, dynamic data stream classification has achieved many successful results through concept drift detection and ensemble learning. However, generally, due to delay in concept drift detection, the active classifier may further learn data belonging to a new concept. This will ultimately degrade the generalization capability of this active classifier on its corresponding concept. Thus, how can a classifier corresponding to one concept unlearns the learned data belonging to another concept? Two unlearning algorithms are proposed to address this problem. The first one based on the passive-aggressive (PA) algorithm adopts the least squares method to reversely update the already-trained model, achieving the effect of approximately unlearning, while another based on a modified PA algorithm achieves complete unlearning by modifying the loss function of the PA algorithm. The comprehensive experiments illustrated the effectiveness of these proposed methods.