Applying Hidden Markov Model for Dynamic Game Balancing

Abstract

In Artificial Intelligence (AI) field, Machine Learning (ML) techniques present an interesting approach for games, where it allows some sort of adaptation along the game session. This adaptation can make games more attractive, avoiding that Non-Player-Characters (NPC) present too easy or hard patterns during the game. In both cases, the player may be frustrated due to undesired experience. Although ML techniques are appealing to be used in games, some games characteristics are hard to model. Besides, there are techniques that require a wide variety of observations, which implies two hard barriers for game application: the first is the power processing to compute a huge amount of data in games, considering the real-time characteristic of this kind of application. The second threat is related to the vast majority of games' attributes that must be described in the model. This work proposes a novel approach using ML technique based on Hidden Markov Model (HMM) for game balancing process. HMM is a powerful technique which can be used to learn patterns based on a strong co-relational between an observation and an unknown variable (the hidden part). Our proposed approach learns the player's pattern based on temporal frame observation by co-relating his/her actions (movements) with game events (NPC destruction). The temporal frame observation approach allows the game to learn about player's pattern even if a different person plays it. After the learning process, the following step is to use the knowledge pattern to adapt the game according to the current player, which normally involves making the game harder for a certain period of time. During this time, another pattern may arise, subjected to be learned. In order to validate the presented approach, a Space Invaders clone has been built, allowing to observe that 54 % of participants had more fun while playing it with ML activated in relation to a base version that did not take into account dynamic difficult balancing.