Robust Training in Multiagent Deep Reinforcement Learning Against Optimal Adversary

Abstract

Industry 5.0 enhances manufacturing ability through efficient human-machine interaction, combining human resources and robots to complete tasks more accurately and effectively. Artificial intelligence (AI) plays an essential role in Industry 5.0. As a branch in AI, multiagent deep reinforcement learning (MADRL) attracts vast attention in both academia and industry. However, there is a gap between virtual and physical environments in terms of how clean an observed state is. In addition, state adversarial attacks can seriously impact the performance of MADRL. Hence, how to improve the robustness of MADRL algorithms is an important research topic. In this article, we propose an optimal policy-based state adversary attack method that would make the MADRL algorithm more robust when it is applied in the training process of agents. Two case studies related to Industry 5.0 and a general case study are presented in which robustness training against the optimal adversarial attack is tested. The MADRL algorithms involved in the experiments include centralized training and decentralized execution (CTDE) framework and shared experience actor-critic (SEAC) to demonstrate the universality of our method.