An Exploratory Analysis of the Multi-Armed Bandit Problem

Abstract

This paper conducts a laboratory experiment to analyze individual behavior in multi-armed bandit problems. Our experiment consists of four types of multi-armed bandit problems: (i) a two-armed indefinite horizon problem, (ii) a two-armed finite horizon problem, (iii) a three-armed indefinite horizon problem, and (iv) a three-armed finite horizon problem. We find that differences in behavior (switching, experimentation, best arm percentage) between these types of multi-armed bandit problems are consistent with predictions. However, we find that subjects use strategies that are different than predicted. We find that commonly suggested deterministic strategies are poor descriptors of subject behavior and that probabilistic strategies better fit the data. In particular, we find that a simple probabilistic ‘win-stay lose-shift’ strategy best fits most subjects.