Online Learning Methods for Networking

In this monograph we provided a tutorial on a family of sequential learning and decision problems known as the multi-armed bandit problems. We introduced a wide range of application scenarios for this learning framework, as well as its many different variants. The more detailed discussion has focused more on the stochastic bandit problems, with rewards driven by either an IID or a Markovian process, and when the environment consists of a single or multiple simultaneous users. We also presented literature on learning of MDPs, which captures coupling among the evolution of different options that a classical MAB problem does not.