Robust Causal Bandits for Linear Models

Abstract

The sequential design of experiments for optimizing a reward function in causal systems can be effectively modeled by the sequential design of interventions in causal bandits (CBs). In the existing literature on CBs, a critical assumption is that the causal models remain constant over time. However, this assumption does not necessarily hold in complex systems, which constantly undergo temporal model fluctuations. This paper addresses the robustness of CBs to such model fluctuations. The focus is on causal systems with linear structural equation models (SEMs). The SEMs and the time-varying pre- and post-interventional statistical models are all unknown. Cumulative regret is adopted as the design criteria, based on which the objective is to design a sequence of interventions that incur the smallest cumulative regret with respect to an oracle aware of the entire causal model and its fluctuations. First, it is established that the existing approaches fail to maintain regret sub-linearity with even a few instances of model deviation. Specifically, when the number of instances with model deviation is as few as $T^{\frac {1}{2L}}$ , where $T$ is the time horizon and $L$ is the length of the longest causal path in the graph, the existing algorithms will have linear regret in $T$ . For instance, when $T=10^{5}$ and $L=3$ , model deviations in 6 out of 105 instances result in a linear regret. Next, a robust CB algorithm is designed, and its regret is analyzed, where upper and information-theoretic lower bounds on the regret are established. Specifically, in a graph with $N$ nodes and maximum degree $d$ , under a general measure of model deviation $C$ , the cumulative regret is upper bounded by $\tilde {\mathcal {O}}\left({d^{L-{}\frac {1}{2}}(\sqrt {NT} + NC)}\right)$ and lower bounded by $\Omega \left({d^{\frac {L}{2}-2}\max \{\sqrt {T}\;, \; d^{2}C\}}\right)$ . Comparing these bounds establishes that the proposed algorithm achieves nearly optimal $\tilde{\mathcal {O}} (\sqrt {T})$ regret when $C$ is $o(\sqrt {T})$ and maintains sub-linear regret for a broader range of $C$ .