Luke Harries, Sebastian Lee, Jaroslaw Rzepecki, Katja Hofmann, Sam Devlin
{"title":"MazeExplorer: A Customisable 3D Benchmark for Assessing Generalisation in Reinforcement Learning","authors":"Luke Harries, Sebastian Lee, Jaroslaw Rzepecki, Katja Hofmann, Sam Devlin","doi":"10.1109/CIG.2019.8848048","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848048","url":null,"abstract":"This paper presents a customisable 3D benchmark for assessing generalisability of reinforcement learning agents based on the 3D first-person game Doom and open source environment VizDoom. As a sample use-case we show that different domain randomisation techniques during training in a key-collection navigation task can help to improve agent performance on unseen evaluation maps.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122914503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Win or Learn Fast Proximal Policy Optimisation","authors":"D. Ratcliffe, Katja Hofmann, Sam Devlin","doi":"10.1109/CIG.2019.8848100","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848100","url":null,"abstract":"AI agents within video games are often required to compete within an environment shared by many other agents. This problem can be tackled by multi-agent reinforcement learning (MARL). One solution to MARL is to learn a Nash Equilibrium Strategy (NES) that guarantees a known minimum payoff when playing against other rational agents. We focus on one approach for learning a NES, Win or Learn Fast (WoLF), WoLF has been shown to converge towards a NES in a variety of matrix-games and grid based games. Research into Deep MARL has focused on performance against opponent agents and with limited quantitative results regarding learning a NES. We present a systematic empirical investigation into the ability of Proximal Policy Optimisation (PPO) to learn a NES, showing instability in certain matrix games. We then present an extension, WoLF-PPO, that is able to learn a policy that is closer to the NES.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125474574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Georgiadis, G. V. Lankveld, Kiavash Bahreini, W. Westera
{"title":"Learning Analytics Should Analyse the Learning: Proposing a Generic Stealth Assessment Tool","authors":"K. Georgiadis, G. V. Lankveld, Kiavash Bahreini, W. Westera","doi":"10.1109/CIG.2019.8847960","DOIUrl":"https://doi.org/10.1109/CIG.2019.8847960","url":null,"abstract":"Stealth assessment could radically extend the scope and impact of learning analytics. Stealth assessment refers to the unobtrusive assessment of learners by exploiting emerging data from their digital traces in electronic learning environments through machine learning technologies. So far, stealth assessment has been studied extensively in serious games, but has not been widely applied, as it is a laborious and complex methodology for which no support tools are available. This study proposes a generic tool for the arrangement of stealth assessment to remove its current limitations and pave the road for its wider adoption. It describes the conceptual design of such a tool including its requirements regarding users, functions, and workflow. A prototype was implemented as a basic console application covering the tool's core requirements, including a Gaussian Naïve Bayes Network utility. Generated input files were used for testing and validating the approach. In a controlled test condition the stealth assessment classification accuracy was found to be inherently stable and high (typically above 92%). It is argued that the proposed approach could radically increase the applicability of stealth assessment in serious games and inform current learning analytics approaches with unobtrusive, more detailed and genuine assessments of learning.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115103285","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
C. Santos, Niels Cornelis Martinus Felicius van Gaans, Vassilis-Javed Khan, P. Markopoulos
{"title":"Effects of advertisements and questionnaire interruptions on the player experience","authors":"C. Santos, Niels Cornelis Martinus Felicius van Gaans, Vassilis-Javed Khan, P. Markopoulos","doi":"10.1109/CIG.2019.8848023","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848023","url":null,"abstract":"New online stores and digital distribution methods have led to the development of alternative monetization models for video-games, such as free-to-play games with advertisements. Although there are many games using such models, until now the effect on the player experience from such interruptions has not been studied. In this controlled experiment, we requested that participants (N=236) play one of three different versions of a platformer game with: 1) no interruptions, 2) 30-second video advertisements, and 3) a multiple-choice questionnaire. We then evaluated the effects on the player experience. The study shows differences in their experiences, namely in: competence, immersion, annoyance, affects, and the reliability of the questionnaire answers. The contribution of this work is to identify which player experience variables are affected by interruptions, which can be valuable for selecting the business model and guiding the game design process.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"176 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114940289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Remixing Headlines for Context-Appropriate Flavor Text","authors":"Judith van Stegeren, M. Theune","doi":"10.1109/CIG.2019.8848050","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848050","url":null,"abstract":"We describe a prototype of Churnalist, a headline generator for creating contextually-appropriate fictional headlines that can be used as flavor text in games. Churnalist creates new headlines by remixing existing headlines. It extracts seed words from free text input, searches for related words in a dataset of word embeddings and uses these words in the new headlines. The system requires no linguistic expertise or hand-coded language models from the user.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"63 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115146055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"When Are We Done with Games?","authors":"Niels Justesen, Michael S. Debus, S. Risi","doi":"10.1109/CIG.2019.8847963","DOIUrl":"https://doi.org/10.1109/CIG.2019.8847963","url":null,"abstract":"From an early point, games have been promoted as important challenges within the research field of Artificial Intelligence (AI). Recent developments in machine learning have allowed a few AI systems to win against top professionals in even the most challenging video games, including Dota 2 and StarCraft. It thus may seem that AI has now achieved all of the long-standing goals that were set forth by the research community. In this paper, we introduce a black box approach that provides a pragmatic way of evaluating the fairness of AI vs. human competitions, by only considering motoric and perceptual fairness on the competitors’ side. Additionally, we introduce the notion of extrinsic and intrinsic factors of a game competition and apply these to discuss and compare the competitions in relation to human vs. human competitions. We conclude that Dota 2 and StarCraft II are not yet mastered by AI as they so far only have been able to win against top professionals in limited competition structures in restricted variants of the games.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120999068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Modelling Player Preferences in AR Mobile Games","authors":"Vivek R. Warriar, John R. Woodward, L. Tokarchuk","doi":"10.1109/CIG.2019.8848082","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848082","url":null,"abstract":"In this paper, we use preference learning techniques to model players’ emotional preferences in an AR mobile game. This exploratory study uses player behaviour to make these preference predictions. The described techniques successfully predict players’ frustration and challenge levels with high accuracy while all other preferences tested (boredom, excitement and fun) perform better than random chance. This paper describes the AR treasure hunt game we developed, the user study conducted to collect player preference data, analysis performed, and preference learning techniques applied to model this data. This work is motivated to personalize players’ experiences by using these computational models to optimize content creation and game balancing systems in these environments. The generality of our technique, limitations, and usability as a tool for personalization of AR mobile games is discussed.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125933859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
T. Cabioch, R. Champagnat, Anne-Gwenn Bosser, Jean-Noël Chiganne, Martín Diéguez
{"title":"Timing Interactive Narratives","authors":"T. Cabioch, R. Champagnat, Anne-Gwenn Bosser, Jean-Noël Chiganne, Martín Diéguez","doi":"10.1109/CIG.2019.8847967","DOIUrl":"https://doi.org/10.1109/CIG.2019.8847967","url":null,"abstract":"Research in Computational Narratives has evidenced the need to provide formal models of narratives integrating action representation together with temporal and causal constraints. Adopting an adequate formalization for narrative actions is critical to the development of generative or interactive systems capable of telling stories whilst ensuring narrative coherence, or dynamic adaptation to user interaction. It may also allow to verify properties of narratives at design time. In this paper, we discuss the issues of interactive story design, verification, and piloting for a specific genre of industrial application, in the field of interactive entertainment: in the games we consider, teams of participants in a Virtual Reality application are guided in real time through a narrative experience by a human storyteller. Like in an escape game, the interactive experience is timed: it should be long enough to provide satisfaction to the players, but come to a conclusion before the game session is over in order to provide closure and a sense of achievement to them. We describe how we integrate narrative time in the story design and use it to the verification of temporal properties of scenarios, building on previous work using Linear Logic and Petri Nets.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125370268","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Kiarostami, Mohammadreza Daneshvaramoli, Saleh Khalaj Monfared, Dara Rahmati, S. Gorgin
{"title":"Multi-Agent non-Overlapping Pathfinding with Monte-Carlo Tree Search","authors":"M. Kiarostami, Mohammadreza Daneshvaramoli, Saleh Khalaj Monfared, Dara Rahmati, S. Gorgin","doi":"10.1109/CIG.2019.8848043","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848043","url":null,"abstract":"In this work, we propose a novel implementation of Monte-Carlo Tree Search (MCTS) algorithm to solve a multiagent pathfinding (MAPF) problem. We employ an optimization of MCTS with low time-complexity and acceptable reliability to approach the MAPF problems with no time constraint. To examine the efficiency and performance of the proposed approach, the NumberLink problem as a MAPF is investigated. We show that the addressed problem could be characterized as multi-agent pathfinding problem with no overlapping paths for the agents. Furthermore, we define this problem to be a simplified and special case of Multi-commodity flow problem (MCFP). Our MCTS solution utilizes a modified search-tree structure to efficiently solve the problem based on a 2-dimensional search space which performs in quadratic time complexity (O(m4) where input size is m2) and linear memory complexity (O(m2)). To evaluate our algorithm, we investigate the efficiency of the proposed solution for the well-known Flow Free puzzle. Our implementation solves a large 40 × 40 Numberlink puzzle in 21 minutes. To the best of our knowledge, there is no other efficient solution for this puzzle where the size of the problem is considerably large.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116108334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ahmed M. Abuzuraiq, Aaron Ferguson, Philippe Pasquier
{"title":"Taksim: A Constrained Graph Partitioning Framework for Procedural Content Generation","authors":"Ahmed M. Abuzuraiq, Aaron Ferguson, Philippe Pasquier","doi":"10.1109/CIG.2019.8848065","DOIUrl":"https://doi.org/10.1109/CIG.2019.8848065","url":null,"abstract":"We present Taksim, an Answer Set Programming (ASP) framework for generating content in games through constrained graph partitioning. We illustrate its expressivity by implementing logical constraints that are relevant to generating the spaces of game levels. Furthermore, we present a case study for creating game levels from a given Mission Graph. Finally, we propose key concepts that make constrained graph partitioning, coupled with ASP, an asset for Procedural Content Generation.","PeriodicalId":177208,"journal":{"name":"2019 IEEE Conference on Games (CoG)","volume":"167 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121704581","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}