FAST: A pioneering unlearning framework integrating fine-tuning, adverse training, and student–teacher methods

Abstract

In the evolving field of machine unlearning, the imperative to protect data privacy while maintaining essential information has become increasingly critical. This paper introduces a pioneering unlearning framework named FAST (Fine-tuning, Adverse Training, and Student–Teacher Methods). FAST is designed to selectively erase privacy-sensitive data while robustly safeguarding valuable information. It employs an innovative integration of the student–teacher architecture, utilizing targeted data and a sophisticated distribution structure refined by the Kullback–Leibler (KL) divergence within the loss function. Enhanced by fine-tuning and adverse training techniques, this integration amplifies beneficial knowledge from competent teachers and reduces ineffective knowledge from less capable ones, thereby enabling more effective and efficient unlearning processes. Furthermore, a new evaluation method called the Unlearning Effectiveness Score (UES) has been proposed for our unlearning model, aimed at providing a comprehensive metric to assess the effectiveness of the unlearning process. This approach rigorously evaluates the model using two sophisticated methods: the Zero Retrain Forgetting (ZRF) metric and Membership Inference Attacks (MIA). The UES is designed to not only measure the effectiveness of forgetting but also to ensure that the model is not easily susceptible to these attacks. These methods facilitate comprehensive comparisons with previous techniques such as bad teaching, Amnesiac, SCRUB, and straightforward fine-tuning. Our thorough experimental analysis, conducted across a variety of deep networks including MobileNet, ResNet, and VGG on the CIFAR and MUFAC datasets, confirms that FAST substantially surpasses existing approaches in both effectively forgetting targeted data and retaining necessary information, as demonstrated by its superior performance on UES metrics.