Selective real-time adversarial perturbations against deep reinforcement learning agents

Abstract

Recent work has shown that deep reinforcement learning (DRL) is vulnerable to adversarial attacks, so that exploiting vulnerabilities in DRL systems through adversarial attack techniques has become a necessary prerequisite for building robust DRL systems. Compared to traditional deep learning systems, DRL systems are characterised by long sequential decisions rather than one-step decision, so attackers must perform multi-step attacks on them. To successfully attack a DRL system, the number of attacks must be minimised to avoid detecting by the victim agent and to ensure the effectiveness of the attack. Some selective attack methods proposed in recent researches, that is, attacking an agent at partial time steps, are not applicable to real-time attack scenarios, although they can avoid detecting by the victim agent. A real-time selective attack method that is applicable to environments with discrete action spaces is proposed. Firstly, the optimal attack threshold T for performing selective attacks in the environment Env is determined. Then, the observation states corresponding to when the value of the action preference function of the victim agent in multiple eposides exceeds the threshold T are added to the training set according to this threshold. Finally, a universal perturbation is generated based on this training set, and it is used to perform real-time selective attacks on the victim agent. Comparative experiments show that our attack method can perform real-time attacks while maintaining the attack effect and stealthiness.