Security Assessment of the Contextual Multi-Armed Bandit - RL Algorithm for Link Adaptation

Abstract

Industry is increasingly adopting Reinforcement Learning algorithms (RL) in production without thoroughly analyzing their security features. In addition to the potential threats that may arise if the functionality of these algorithms is compromised while in operation. One of the well-known RL algorithms is the Contextual Multi-Armed Bandit (CMAB) algorithm. In this paper, we explore how the CMAB can be used to solve the Link Adaptation problem – a well-known problem in the telecommunication industry by learning the optimal transmission parameters that will maximize a communication link’s throughput. We analyze the potential vulnerabilities of the algorithm and how they may adversely affect link parameters computation. Additionally, we present a provable security assessment for the Contextual Multi-Armed Bandit Reinforcement Learning (CMAB-RL) algorithm in a network simulated environment using Ray. This is by demonstrating CMAB security vulnerabilities theoretically and practically. Some security controls are proposed for CMAB agent and the surrounding environment. In order to fix those vulnerabilities and mitigate the risk. These controls can be applied to other RL agents in order to design more robust and secure RL agents.