Simulation-Driven Balancing of Competitive Game Levels With Reinforcement Learning

Abstract

The balancing process for game levels in competitive two-player contexts involves a lot of manual work and testing, particularly for nonsymmetrical game levels. In this work, we frame game balancing as a procedural content generation task and propose an architecture for automatically balancing of tile-based levels within the procedural content generation via reinforcement learning framework (PCGRL) framework. Our architecture is divided into three parts: first, a level generator, second, a balancing agent, and third, a reward modeling simulation. Through repeated simulations, the balancing agent receives rewards for adjusting the level toward a given balancing objective, such as equal win rates for all players. To this end, we propose new swap-based representations to improve the robustness of playability, thereby enabling agents to balance game levels more effectively and quickly compared to traditional PCGRL. By analyzing the agent's swapping behavior, we can infer which tile types have the most impact on the balance. We validate our approach in the neural massively multiplayer online environment in a competitive two-player scenario. In this article, we present improved results, explore the applicability of the method to various forms of balancing beyond equal balancing, compare the performance to another search-based approach, and discuss the application of existing fairness metrics to game balancing.