Defending against attacks in deep learning with differential privacy: a survey

Recently, we have witnessed the revolutionary development of deep learning. As the application domain of deep learning has expanded, its privacy risks have attracted attention since deep leaning methods often use private data for training. Some methods for attacking deep learning, such as membership inference attacks, increase the privacy risks of deep learning models. One risk-reducing defensive strategy with great potential is to apply some degree of random perturbation during the training (or other) phase. Therefore, differential privacy, as a privacy protection framework originally designed for publishing data, is widely used to protect the privacy of deep learning models due to its solid mathematical foundation. In this paper, we first introduce several attack methods that threaten deep learning. Then, we systematically review the cross-applications of differential privacy and deep learning to protect deep learning models. We encourage researchers to visually demonstrate the defense effects of their approaches in the literature rather than solely providing rigorous mathematical proofs. In addition to privacy, we also discuss and review the impact of differential privacy on the robustness, overfitting, and fairness of deep neural networks. Finally, we analyze some potential future research directions, highlighting the significant potential for differential privacy to make positive contributions to future deep learning systems.