Privacy preservation in deep reinforcement learning: A training perspective

Abstract

Reinforcement learning (RL) is a principled AI framework for autonomous, experience-driven learning. Deep reinforcement learning (DRL) enhances this by incorporating deep learning models, promoting a higher-level understanding of the visual world. However, privacy concerns are emerging in RL applications that involve vast amounts of private information. Recent studies have demonstrated that DRL can leak private information and be vulnerable to attacks aiming to infer the training environment from an agent’s behaviors without direct access to the environment. To address these privacy concerns, we propose a differentially private DRL approach that obfuscates the agent’s observations from each visited state. This defends against privacy leakage attacks and prevents the inference of the agent’s training environment from its optimized policy. We provide a theoretical analysis and design comprehensive experiments to thoroughly reproduce the privacy leakage attack. Both the theoretical analysis and experimental results demonstrate that our method effectively defends against privacy leakage attacks while maintaining the model utility of the RL agent.