Stochastic Games with Sensing Costs

Abstract

In real-world games involving autonomous agents making decisions under uncertainty [1], the agents are often subject to sensing and communication limitations. In these cases, it is desirable to win the game, while also minimizing an agent’s sensing budget. In particular, in two-player uncertain adversarial environments, where one player enters the opponent’s territory, we seek a wining strategy with minimum sensing. In this paper, we consider finite two-player stochastic games, wherein in addition to the conventional cost over states and actions of each player, we include the sensing budget in terms of transfer entropy. We find a set of pure and mixed strategies for such a game via dynamic programming. The application of dynamic programming leads to a set of coupled nonlinear equations that we solve using the modified Arimoto-Blahut algorithm. The efficacy of the proposed method is illustrated by a stochastic unmanned aerial vehicle (UAV) pursuit-evasion game example using the tool AMASE.