Byzantine-Resilient Distributed Bandit Online Optimization in Dynamic Environments

Abstract

We consider the constrained multi-agent online optimization problem in dynamic environments that are vulnerable to Byzantine attacks, where some infiltrated agents may deviate from the prescribed update rule and send arbitrary messages. The objective functions are exposed in a bandit form, i.e., only the function value is revealed to each agent at the sampling instance, and held privately by each agent. The agents only exchange information with their neighbors to update decisions, and the collective goal is to minimize the sum of the unattacked agents' objective functions in dynamic environments, where the same function can only be sampled once. To handle this problem, a Byzantine-Resilient Distributed Bandit Online Convex Optimization (BR-DBOCO) algorithm that can tolerate up to $\mathcal {B}$ Byzantine agents is developed. Specifically, the BR-DBOCO employs the one-point bandit feedback (OPBF) mechanism and state filter to cope with the objective function, which cannot be explicitly expressed in dynamic environments and the arbitrary deviation states caused by Byzantine attacks, respectively. We show that sublinear expected regret is achieved if the accumulative deviation of the comparator sequence also grows sublinearly with a proper exploration parameter. Finally, experimental results are presented to illustrate the effectiveness of the proposed algorithm.