Asymptotic analysis of large population stochastic games

Abstract

We study stochastic games with a large number of players, where players are coupled via their payoff functions. A standard solution concept for such games is Markov perfect equilibrium (MPE). It is well known that the computation of MPE suffers from the "curse of dimensionality." To deal with this complexity, several researchers have introduced a notion of mean field equilibrium that we call oblivious equilibrium (OE). In OE, each player reacts to only the average behavior of other players. In this paper, we develop a unified framework to study OE in large population stochastic games. In particular, we prove that under a set of simple assumptions on the model, an OE always exists. Furthermore, as a simple consequence of this existence theorem, we prove that OE approximates MPE well: we show that from the viewpoint of a single agent, a near optimal decision making policy is one that reacts only to the average behavior of its environment. We also study two different classes for games, competition and coordination games. For these classes of games, we isolate key assumptions on the model primitives under which OE exists and approximates MPE asymptotically.