Passivity, RL and Learning in Multi-Agent Games

Abstract

Learning algorithm behavior highly depends on the game setting. In this tutorial talk, we discuss how these dependencies can be explained, if one regards them through a passivity lens. We focus on two representative instances in reinforcement learning: payoff-based play, and Q-learning. We show how one can exploit geometric features of different classes of games, together with dissipativity/passivity properties of interconnected systems to guarantee global convergence to a Nash equilibrium. Besides simplifying the proof of convergence, one can generate algorithms that work for classes of games with less stringent assumptions, by using passivity and basic properties of interconnected systems.