Leveraging deep reinforcement learning for dynamic NPC behavior and enhanced player experience in unity3d

Abstract

With the innovation of information technology and the improvement of computer hardware and software, players’ needs gradually change from pursuing the ultimate visual and auditory feast to the inner performance, gameplay, interactive elements, etc. As an essential part of game content, game AI plays the role of communication and interaction with players. Since the emergence of game AI, it has been paid attention to by game developers. However, it is also a difficult job to make brilliant game AI. The common approaches to implementing game AI are finite state machines and behavior trees. Still, these two approaches require much work to implement flexible game AI and are difficult to maintain later. Therefore, this paper aims to investigate machine learning to train a compliant and flexible game AI. Specifically, this paper adopts a method to train game AI in Unity scenes using machine learning methods such as deep reinforcement learning with the help of the ML-Agents toolkit and Python programming interface. In order to better test the performance of the machine learning method, this study designs a game based on the Unity game engine, which includes a game AI implemented using behavior trees and machine learning. Through the production process and the final implementation results, this paper compares the differences in design ideas and implementation process between using behavior trees and using machine learning to implement game AI, as well as the advantages and disadvantages of each. The purpose of this study is to promote machine learning in the field of game research and achieve higher operational efficiency. Relevant existing principles and guidelines inform the game design process. In addition, the game proposed in this paper can be used as future research to promote machine learning in games to achieve a more efficient, more straightforward design and better player experience. After the game implementation, a quantitative research method is used to measure the players’ immersion in the game and the players’ satisfaction with the designed game AI. The evaluation results showed that most respondents believed that the proposed agents performed well against both human players and inbuilt game agents.