Balancing the Performance of a FightingICE Agent using Reinforcement Learning and Skilled Experience Catalogue

Abstract

Dynamic Difficulty Adjustment (DDA) is the process of changing the challenge offered dynamically based on the player's performance, as opposed to the player manually choosing the difficulty from a set of options. This helps in alleviating player frustration by having the opponents' skill match that of the player's. In this work, we propose a novel application of a DDA technique called Skilled Experience Catalogue (SEC) which has previously been used with success in First Person Shooter games. This approach uses experiential milestones of the learning process of an agent trained using Reinforcement Learning (RL). We have designed and implemented a custom SEC on top of the FightingICE platform that is used in the Fighting Game Artificial Intelligence (FTGAI) competition. We deployed our SEC agent against three fixed-strategy opponents and showed that we could successfully balance the game-play in two out of the three opponents over 150 games against each. Balancing was not achieved against the third opponent since the RL agent could not reach the required skill level after its initial training.