Revisiting Machine Learning Training Process for Enhanced Data Privacy

Abstract

The increasing use of machine learning algorithms for nearly every aspect of our lives has brought a new challenge to the forefront, one of user-privacy. Once the data has been shared by the user online, it is difficult to revoke the access of that data if it has already been used to train the model. For any personal data, every user should reserve the right for the data to be forgotten. To solve the above-mentioned problem, a few frameworks have been introduced recently to achieve machine unlearning or inverse learning. Although there is no specific definition of forgetting in DNNs (deep neural networks) yet, our focus will be on selectively forgetting a subset of data belonging to a class, which was initially used to train the model, without the need of re-training from scratch, nor using the initial training data. This method scrubs the weights clean of the data that needs to be forgotten. Concepts for the stability of stochastic gradient descent and differential privacy are exploited in this approach to address the problem of selective forgetting in DNNs.