Learning in games: a systematic review

Game theory studies the mathematical models for self-interested individuals. Nash equilibrium is arguably the most central solution in game theory. While finding the Nash equilibrium in general is known as polynomial parity arguments on directed graphs (PPAD)-complete, learning in games provides an alternative to approximate Nash equilibrium, which iteratively updates the player’s strategy through interactions with other players. Rules and models have been developed for learning in games, such as fictitious play and no-regret learning. Particularly, with recent advances in online learning and deep reinforcement learning, techniques from these fields greatly boost the breakthroughs in learning in games from theory to application. As a result, we have witnessed many superhuman game AI systems. The techniques used in these systems evolve from conventional search and learning to purely reinforcement learning (RL)-style learning methods, gradually getting rid of the domain knowledge. In this article, we systematically review the above techniques, discuss the trend of basic learning rules towards a unified framework, and recap applications in large games. Finally, we discuss some future directions and make the prospect of future game AI systems. We hope this article will give some insights into designing novel approaches.