Variants of Harsanyi’s tracing procedures for selecting a perfect stationary equilibrium in stochastic games

Abstract

The concept of perfect stationary equilibrium (PSE) is defined to eliminate some counterintuitive subgame perfect equilibria in stationary strategies (SSPEs) for stochastic games. While computational tools are vital to the practical applications of stochastic games, there are limited methods available for computing PSEs in the existing literature. A stochastic version of Harsanyi’s linear tracing procedure (SLTP) was developed by Herings and Peeters for selecting an SSPE. Nonetheless, their method fails to find a PSE directly. In this paper, we develop a variant of the SLTP by formulating a perturbed stochastic game in which each player maximizes her payoff in a state against a convex linear combination of a prior-belief profile and a mixed strategy profile of other players. Furthermore, by integrating logarithmic-barrier terms into the payoff functions of the perturbed game, we formulate a stochastic version of Harsanyi’s logarithmic tracing procedure (SLogTP) and then develop a variant of the SLogTP. We prove the variants of the SLTP and SLogTP globally converge to a PSE for any stochastic game. Extensive numerical experiments are carried out, and the numerical results illustrate the effectiveness and efficiency of the proposed methods.