2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)最新文献

Applying Hidden Markov Model for Dynamic Game Balancing 隐马尔可夫模型在动态博弈平衡中的应用

2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) Pub Date : 2020-11-01 DOI: 10.1109/SBGames51465.2020.00016

M. Zamith, José Ricardo da Silva, E. Clua, M. Joselli

{"title":"Applying Hidden Markov Model for Dynamic Game Balancing","authors":"M. Zamith, José Ricardo da Silva, E. Clua, M. Joselli","doi":"10.1109/SBGames51465.2020.00016","DOIUrl":"https://doi.org/10.1109/SBGames51465.2020.00016","url":null,"abstract":"In Artificial Intelligence (AI) field, Machine Learning (ML) techniques present an interesting approach for games, where it allows some sort of adaptation along the game session. This adaptation can make games more attractive, avoiding that Non-Player-Characters (NPC) present too easy or hard patterns during the game. In both cases, the player may be frustrated due to undesired experience. Although ML techniques are appealing to be used in games, some games characteristics are hard to model. Besides, there are techniques that require a wide variety of observations, which implies two hard barriers for game application: the first is the power processing to compute a huge amount of data in games, considering the real-time characteristic of this kind of application. The second threat is related to the vast majority of games' attributes that must be described in the model. This work proposes a novel approach using ML technique based on Hidden Markov Model (HMM) for game balancing process. HMM is a powerful technique which can be used to learn patterns based on a strong co-relational between an observation and an unknown variable (the hidden part). Our proposed approach learns the player's pattern based on temporal frame observation by co-relating his/her actions (movements) with game events (NPC destruction). The temporal frame observation approach allows the game to learn about player's pattern even if a different person plays it. After the learning process, the following step is to use the knowledge pattern to adapt the game according to the current player, which normally involves making the game harder for a certain period of time. During this time, another pattern may arise, subjected to be learned. In order to validate the presented approach, a Space Invaders clone has been built, allowing to observe that 54 % of participants had more fun while playing it with ML activated in relation to a base version that did not take into account dynamic difficult balancing.","PeriodicalId":335816,"journal":{"name":"2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)","volume":"88 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126027423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

AI4U: A Tool for Game Reinforcement Learning Experiments AI4U:游戏强化学习实验的工具

2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) Pub Date : 2020-11-01 DOI: 10.1109/SBGames51465.2020.00014

Gilzamir Gomes, C. Vidal, J. B. C. Neto, Y. L. Nogueira

{"title":"AI4U: A Tool for Game Reinforcement Learning Experiments","authors":"Gilzamir Gomes, C. Vidal, J. B. C. Neto, Y. L. Nogueira","doi":"10.1109/SBGames51465.2020.00014","DOIUrl":"https://doi.org/10.1109/SBGames51465.2020.00014","url":null,"abstract":"Reinforcement Learning is a promising approach to the design of Non-Player Characters (NPCs). It is challenging, however, to design games enabled to support reinforcement learning because, in addition to specifying the environment and the agent that controls the character, there is the challenge of modeling a significant reward function for the expected behavior from a virtual character. To alleviate the challenges of this problem, we have developed a tool that allows one to specify, in an integrated way, the environment, the agent, and the reward functions. The tool provides a visual and declarative specification of the environment, providing a graphic language consistent with game events. Besides, it supports the specification of non-Markovian reward functions and is integrated with a game development platform that makes it possible to specify complex and interesting environments. An environment modeled with this tool supports the implementation of most current state-of-the-art reinforcement learning algorithms, such as Proximal Policy Optimization and Soft Actor-Critic algorithms. The objective of the developed tool is to facilitate the experimentation of learning in games, taking advantage of the existing ecosystem around modern game development platforms. Applications developed with the support of this tool show the potential for specifying game environments to experiment with reinforcement learning algorithms.","PeriodicalId":335816,"journal":{"name":"2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)","volume":"102 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124716146","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Towards the Design of Adaptive Virtual Reality Horror Games: A Model of Players' Fears Using Machine Learning and Player Modeling 面向自适应虚拟现实恐怖游戏的设计:使用机器学习和玩家建模的玩家恐惧模型

2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) Pub Date : 2020-11-01 DOI: 10.1109/SBGames51465.2020.00031

E. S. D. Lima, Bruno M. C. Silva, Gabriel Teixeira Galam

引用次数: 1

Investigating Case Learning Techniques for Agents to Play the Card Game of Truco 基于案例学习技术的智能体Truco纸牌游戏研究

2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) Pub Date : 2020-11-01 DOI: 10.1109/SBGames51465.2020.00024

Ruan C. B. Moral, G. B. Paulus, J. Assunção, L. A. L. Silva

引用次数: 1

Drafting in Collectible Card Games via Reinforcement Learning 基于强化学习的收集卡牌游戏制图

2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) Pub Date : 2020-11-01 DOI: 10.1109/SBGames51465.2020.00018

R. Vieira, A. Tavares, L. Chaimowicz

引用次数: 4

An Intelligent Agent Playing Generic Action Games based on Deep Reinforcement Learning with Memory Restrictions 基于记忆限制下深度强化学习的通用动作游戏智能体

2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) Pub Date : 2020-11-01 DOI: 10.1109/SBGames51465.2020.00015

Lucas Antunes de Almeida, M. Thielo

{"title":"An Intelligent Agent Playing Generic Action Games based on Deep Reinforcement Learning with Memory Restrictions","authors":"Lucas Antunes de Almeida, M. Thielo","doi":"10.1109/SBGames51465.2020.00015","DOIUrl":"https://doi.org/10.1109/SBGames51465.2020.00015","url":null,"abstract":"Among the topics that increasingly gained special attention in Computer Science recently, the evolution of Artificial Intelligence has been one of the most prominent subjects, especially when related to games. In this work we developed an intelligent agent with memory restrictions so to investigate its ability to learn playing multiple, different games without the need of being provided with specific details for each of the games. As a measure of quality of the agent, we used the difference between its score and the scores obtained by casual human players. Aiming to address the possibilities of using Deep Learning for General Game Playing in less powerful devices, we explicitly limited the amount of memory available for the agent, apart from the commonly used physical memory limit for most works in the area. For the abstraction of machine learning and image processing stages, we used the Keras and Gym libraries. As a result, we obtained an agent capable of playing multiple games without the need to provide rules in advance, but receiving at each moment only the game video frame, the current score and whether the current state represents an endgame. To assess the agent effectiveness, we submitted it to a set of Atari 2600™ games, where the scores obtained were compared to casual human players and discussed. In the conclusion, we show that promising results were obtained for these games even with memory limitations and finally a few improvements are proposed.","PeriodicalId":335816,"journal":{"name":"2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)","volume":"62 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128303541","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

URNAI: A Multi-Game Toolkit for Experimenting Deep Reinforcement Learning Algorithms URNAI:一个用于实验深度强化学习算法的多游戏工具包

2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) Pub Date : 2020-11-01 DOI: 10.1109/SBGames51465.2020.00032

Marco A. S. Araùjo, L. P. Alves, C. Madeira, Marcos M. Nóbrega

{"title":"URNAI: A Multi-Game Toolkit for Experimenting Deep Reinforcement Learning Algorithms","authors":"Marco A. S. Araùjo, L. P. Alves, C. Madeira, Marcos M. Nóbrega","doi":"10.1109/SBGames51465.2020.00032","DOIUrl":"https://doi.org/10.1109/SBGames51465.2020.00032","url":null,"abstract":"In the last decade, several game environments have been popularized as testbeds for experimenting reinforcement learning algorithms, an area of research that has shown great potential for artificial intelligence based solutions. These game environments range from the simplest ones like CartPole to the most complex ones such as StarCraft II. However, in order to experiment an algorithm in each of these environments, researchers need to prepare all the settings for each one, a task that is very time consuming since it entails integrating the game environment to their software and treating the game environment variables. So, this paper introduces URNAI, a new multi-game toolkit that enables researchers to easily experiment with deep reinforcement learning algorithms in several game environments. To do this, URNAI implements layers that integrate existing reinforcement learning libraries and existing game environments, simplifying the setup and management of several reinforcement learning components, such as algorithms, state spaces, action spaces, reward functions, and so on. Moreover, URNAI provides a framework prepared for GPU supercomputing, which allows much faster experiment cycles. The first toolkit results are very promising.","PeriodicalId":335816,"journal":{"name":"2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130442195","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Development of an Autonomous Agent based on Reinforcement Learning for a Digital Fighting Game 基于强化学习的数字格斗游戏自主代理的开发

2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) Pub Date : 2020-11-01 DOI: 10.1109/SBGames51465.2020.00017

J. R. Bezerra, L. F. Góes, Alysson Ribeiro Da Silva

引用次数: 1

A Baseline Approach for Goalkeeper Strategy using Sarsa with Tile Coding on the Half Field Offense Environment 半场进攻环境下基于Sarsa的守门员策略基线分析

2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) Pub Date : 2020-11-01 DOI: 10.1109/SBGames51465.2020.00012

V. G. F. Barbosa, R. Neto, Roberto V. L. Gomes Rodrigues

引用次数: 1

[Title page] (标题页)

2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) Pub Date : 2020-11-01 DOI: 10.1109/sbgames51465.2020.00001

引用次数: 0