Marco Pleines, Frank Zimmer, Vincent-Pierre Berges
{"title":"Action Spaces in Deep Reinforcement Learning to Mimic Human Input Devices","authors":"Marco Pleines, Frank Zimmer, Vincent-Pierre Berges","doi":"10.1109/CIG.2019.8848080","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848080","url":null,"abstract":"Enabling agents to generally play video games requires to implement a common action space that mimics human input devices like a gamepad. Such action spaces have to support concurrent discrete and continuous actions. To solve this problem, this work investigates three approaches to examine the application of concurrent discrete and continuous actions in Deep Reinforcement Learning (DRL). One approach implements a threshold to discretize a continuous action, while another one divides a continuous action into multiple discrete actions. The third approach creates a multiagent to combine both action kinds. These approaches are benchmarked by two novel environments. In the first environment (Shooting Birds) the goal of the agent is to accurately shoot birds by controlling a cross-hair. The second environment is a simplification of the game Beastly Rivals On-slaught, where the agent is in charge of its controlled character’s survival. Throughout multiple experiments, the bucket approach is recommended, because it is trained faster than the multiagent and is more stable than the threshold approach. Due to the contributions of this paper, consecutive work can start training agents using visual observations.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"66 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124489847","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Predicting the monetization percentage with survival analysis in free-to-play games","authors":"Riikka Numminen, Markus Viljanen, T. Pahikkala","doi":"10.1109/CIG.2019.8848045","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848045","url":null,"abstract":"Understanding and predicting player monetization is very important, because the free-to-play revenue model is so common. Many game developers now face a new challenge of getting users to buy in the game rather than getting users to buy the game. In this paper, we present a method to predict what percentage of all players will eventually monetize for a limited follow-up game data set. We assume that the data is described by a survival analysis based cure model, which can be applied to unlabeled data collected from any free-to-play game. The model has latent variables, so we solve the optimal parameters of the model with the Expectation Maximization algorithm. The result is a simple iterative algorithm, which returns the estimated monetization percentage and the estimated monetization rate in the data set.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"40 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127900139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evolution of Kiting Behavior in a Two Player Combat Problem","authors":"Pavlos Androulakakis, Zachariah E. Fuchs","doi":"10.1109/CIG.2019.8848124","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848124","url":null,"abstract":"We examine the use of an evolutionary algorithm to design a feedback controller for a two player combat problem. This problem consists of two players on a one-dimensional line. One player is referred to as the Defender and is held at the origin (unable to move). The other player is referred to as the Attacker and is free to move back and forth with a constant speed. The goal of the Attacker is to inflict as much damage as it can on the Defender while suffering as little damage as possible. The greater the difference between damage inflicted vs damage taken, the more successful the attack. The Attacker’s controller is represented by a parameterized control vector. An evolutionary algorithm is then used to evolve populations of these control vectors in an attempt to obtain a near optimal feedback controller.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131918384","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mitchell Miller, Noah Paige, Guraik Clair, C. Eckhardt
{"title":"An Analysis of Peer Presence Social Group Dynamics to Enhance Player Engagement in Multiplayer Games","authors":"Mitchell Miller, Noah Paige, Guraik Clair, C. Eckhardt","doi":"10.1109/CIG.2019.8848078","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848078","url":null,"abstract":"In this paper, we seek to identify game design paradigms that enhance the player experience for multiplayer games in regards to engagement and immersion through social group dynamic emphasized level design based on peer presence. In order to identify these paradigms, we developed a 2D platform game consisting of several scenarios designed to invoke different sub-categories of peer presence social group dynamics such as group cohesion, peer pressure, leader-observer social identity role distribution and black sheep effects. We argue that risk and reward core game mechanics can be carried out or reinforced by the social group dynamics phenomena. A group of participants was monitored through gameplay, and administered surveys prior to, as well as following the game experience.Our study successfully found game design patterns which highlight specific behaviors within group dynamics, especially group cohesion and time-sensitive decision making with regards to peer pressure, which, compared to the single-player control group, increase players engagement and immersion during multiplayer gameplay. Our investigations into these player behavior patterns aim to help game designers enhance player experience in multiplayer games.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132976180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Hyperstate Space Graphs for Automated Game Analysis","authors":"Michael Cook, Azalea Raad","doi":"10.1109/CIG.2019.8848026","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848026","url":null,"abstract":"Automatically analysing games is an important challenge for automated game design, general game playing, and co-creative game design tools. However, understanding the nature of an unseen game is extremely difficult due to the lack of a priori design knowledge and heuristics. In this paper we formally define hyperstate space graphs, a compressed form of state space graphs which can be constructed without any prior design knowledge about a game. We show how hyperstate space graphs produce compact representations of games which closely relate to the heuristics designed by hand for search-based AI agents; we show how hyperstate space graphs also relate to modern ideas about game design; and we point towards future applications for hyperstates across game AI research.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"94 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128849999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Characteristics Study of Dance-charts on Rhythm-based Video Games","authors":"Yudai Tsujino, Ryosuke Yamanishi, Y. Yamashita","doi":"10.1109/CIG.2019.8848126","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848126","url":null,"abstract":"This paper studies the characteristics of dancecharts on rhythm-based video games while clustering the dancecharts with the track features. In the opinion of expert players, it is considered that the difficulty level of the rhythm-based games depends on multiple features for the song and chart. The difficulty levels for dance games has not well been studied though it has been already done for other genres of games. This paper designed the track features for dance games, which would be varied depending on characteristics of the track. Moreover, we conducted a dance-charts clustering by using the k-means method with the designed features. As a result of the clustering, it was confirmed that the clusters were composed mostly based on the step frequency and the complexity of rhythm. Also, it was found that the scores with the extremely characteristic features were separated from other scores and clustered as one class with themselves. The discussion in this paper indicated a possibility for the objective evaluation of the dance-charts.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134499009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Realtime Adaptive Virtual Reality for Pain Reduction","authors":"A. Dingli, Luca Bondin","doi":"10.1109/CIG.2019.8848119","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848119","url":null,"abstract":"Recent years have seen digital game mediums taking conventional amusement, entertainment and leisure industries by storm. They have revolutionized the system to the extent that the industry cannot now even dream to do without this overwhelming reality. The same game mediums that have capitalized on intrinsic leisure aspects have simultaneously focused with equal vigor on other equally, if not more, important collateral objectives. This paper builds on this concept and discusses a work in progress currently being carried out at the University of Malta. It proposes the use of games as a means of distraction therapy for individuals undergoing painful clinical treatment procedures. The creation of an adaptive Virtual Reality (VR) game within an Artificial Intelligence framework will without doubt be of a significantly greater benefit to the community than mere entertainment applications.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114718958","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sarjak Thakkar, Changxing Cao, Lifan Wang, Tae Jong Choi, J. Togelius
{"title":"Autoencoder and Evolutionary Algorithm for Level Generation in Lode Runner","authors":"Sarjak Thakkar, Changxing Cao, Lifan Wang, Tae Jong Choi, J. Togelius","doi":"10.1109/CIG.2019.8848076","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848076","url":null,"abstract":"Procedural content generation can be used to create arbitrarily large amounts of game levels automatically, but traditionally the PCG algorithms needed to be developed or adapted for each game manually. Procedural Content Generation via Machine Learning (PCGML) harnesses the power of machine learning to semi-automate the development of PCG solutions, training on existing game content so as to create new content from the trained models. One of the machine learning techniques that have been suggested for this purpose is the autoencoder. However, very limited work has been done to explore the potential of autoencoders for PCGML. In this paper, we train autoencoders on levels for the platform game Lode Runner, and use them to generate levels. Compared to previous work, we use a multi-channel approach to represent content in full fidelity, and we compare standard and variational autoencoders. We also evolve the values of the hidden layer of trained autoencoders in order to find levels with desired properties.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115102532","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Guntis Barzdins, D. Gosko, Paulis F. Barzdins, Uldis Lavrinovics, Gints Bernans, E. Celms
{"title":"RDF* Graph Database as Interlingua for the TextWorld Challenge","authors":"Guntis Barzdins, D. Gosko, Paulis F. Barzdins, Uldis Lavrinovics, Gints Bernans, E. Celms","doi":"10.1109/CIG.2019.8848012","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848012","url":null,"abstract":"This paper briefly describes the top-scoring submission to the First TextWorld Problems: A Reinforcement and Language Learning Challenge. To alleviate the partial observability problem, characteristic to the TextWorld games, we split the Agent into two independent components: Observer and Actor, communicating only via the Interlingua of the RDF* graph database. The RDF* graph database serves as the “world model” memory incrementally updated by the Observer via FrameNet informed Natural Language Understanding techniques and is used by the Actor for the efficient exploration and planning of the game Action sequences. We find that the deep-learning approach works best for the Observer component while the Actor policy is better served by backtracking over the set of rules.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122063987","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Searching the Latent Space of a Generative Adversarial Network to Generate DOOM Levels","authors":"Edoardo Giacomello, P. Lanzi, D. Loiacono","doi":"10.1109/CIG.2019.8848011","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848011","url":null,"abstract":"In this work, following the same approach successfully applied to evolve Super Mario levels, we applied the CMA-ES to search the latent space of a GAN previously trained to generate DOOM levels. Combining a search algorithm with a model trained in a supervised setting, allows to take advantage from both these paradigms. From one hand, the GAN is able to generate contents exploiting the design patterns learned from all the examples it was trained from. On the other hand, the CMA-ES can effectively search this design space for specific contents that meet some given design objectives. In particular, we tested our approach evolving three very different type of levels: an arena level (i.e., few large areas), a labyrinth level (i.e., many corridors and small areas), and a complex level (i.e., a balanced mix of large and small areas). Our results show that the latent space of a GAN can be effectively searched by the CMA-ES to find DOOM levels that fit accurately the objectives but, at the same time, are also novel.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"373 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123952775","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}