Markov Quantal Response Equilibrium and a Homotopy Method for Computing and Selecting Markov Perfect Equilibria of Dynamic Stochastic Games

Abstract

We formally define Markov quantal response equilibrium (QRE) and prove existence for all finite discounted dynamic stochastic games. The special case of logit Markov QRE constitutes a mapping from precision parameter λ to sets of logit Markov QRE. The limiting points of this correspondence are shown to be Markov perfect equilibria. Furthermore, the logit Markov QRE correspondence can be given a homotopy interpretation. We prove that for all games, this homotopy contains a branch connecting the unique solution at λ = 0 to a unique limiting Markov perfect equilibrium. This result can be leveraged both for the computation of Markov perfect equilibria, and also as a selection criterion.