Procedural Game Adaptation: Framing Experience Management as Changing an MDP

In this paper, we present the Procedural Game Adaptation (PGA) framework, a designer-controlled way to change a game's dynamics during end-user play. We formalize a video game as a Markov Decision Process, and frame the problem as maximizing the reward of a given player by modifying the game's transition function. By learning a model of each player to estimate her rewards, PGA managers can change the game's dynamics in a player-informed way. Following a formal definition of the components of the framework, we illustrate its versatility by using it to represent two existing adaptive systems: PaSSAGE, and Left 4 Dead's AI Director.