Some algorithms for correlated bandits with non-stationary rewards: Regret bounds and applications

Abstract

We first propose an online learning model wherein rewards for different actions/arms used by the user can be correlated and the reward stream can be non-stationary. Thus, this extends the standard multi-armed bandit learning model. We propose two algorthims, Greedy and Regression based UCB, that attempt to minimize the expected regret. We also obtain non-trivial upper bounds for the expected regret through theoretical analysis. We also provide some evidence for sub-polynomial increase in expected regret upon appropriate tuning of algorithm input parameters. These models are motivated by the problem of dynamic pricing of a product faced by a typical online retailer.