{"title":"World Models with an Entity-Based Representation","authors":"Nazanin Yousefzadeh Khameneh, Matthew J. Guzdial","doi":"10.1609/aiide.v18i1.21966","DOIUrl":"https://doi.org/10.1609/aiide.v18i1.21966","url":null,"abstract":"Reinforcement learning (RL) is a powerful way to solve sequential decision-making tasks. However training an RL agent in a complex environment requires a large amount of interactions, which is non-ideal when acting in an environment is costly or dangerous. One alternative is to learn an approximation of the real environment, referred to as a world model. This simulator can be used to train an agent and transfer the learned policy to the real environment. Unfortunately, training world models have traditionally required a significant number of interactions in the real environment. This brings us to the same problem when it is costly or dangerous to act in the real environment. To address this problem, we present an entity-based representation and corresponding architecture, which allows for greater data efficiency in world model training. Our approach outperforms other world model baselines in an initial application to the game Pong.","PeriodicalId":92576,"journal":{"name":"Proceedings. AAAI Artificial Intelligence and Interactive Digital Entertainment Conference","volume":"23 1 1","pages":"215-222"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78446323","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Step: A Highly Expressive Text Generation Language","authors":"I. Horswill","doi":"10.1609/aiide.v18i1.21969","DOIUrl":"https://doi.org/10.1609/aiide.v18i1.21969","url":null,"abstract":"Games often generate text from human-authored templates and adapt that text to relevant context. Many systems have been developed to aid this process using tech-niques such as context-free grammars, randomization, logic programming, global state, and HTN planning.\u0000 In this paper, I present Step, a novel programming language for text generation. So far as I can determine, previous techniques can all be implemented within Step using a few lines of code. This allows designers to mix-and-match features as needed, without having to write an interpreter for a new language.\u0000 While extremely expressive as a programming language, Step is also intended to allow writers to add new text to an existing system with minimal markup and little knowledge of programming. In a head-to-head comparison, the Step implementation was more compact than, and used less markup than, a Prolog implementation of the same generator.","PeriodicalId":92576,"journal":{"name":"Proceedings. AAAI Artificial Intelligence and Interactive Digital Entertainment Conference","volume":"16 1","pages":"240-249"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87702024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Nonsense Laboratory","authors":"Allison Parrish, Jenny Goldstick, Tim Szetela","doi":"10.1609/aiide.v18i1.21978","DOIUrl":"https://doi.org/10.1609/aiide.v18i1.21978","url":null,"abstract":"We present the Nonsense Laboratory, a series of playful web-based tools for manipulating the way words sound and the way words are spelled. The Laboratory's various playful interfaces give users an opportunity to adjust, poke at, mangle, curate, rearrange and elaborate the sounds of words in written text. The project makes use of Pincelate, a code library and machine learning model for phoneme-to-grapheme and grapheme-to-phoneme tasks in English, trained on the CMU Pronouncing Dictionary. The goal of the project is twofold: first, to make possible new ways of manipulating spelling (akin to playing a musical instrument or modeling with clay); and second, to demystify machine learning by providing an intuitive, friendly interface to a machine learning model. We briefly describe the design and implementation of the Nonsense Laboratory's five interfaces, and situate the project in our own research concerning design, literary studies, linguistics, and machine learning.","PeriodicalId":92576,"journal":{"name":"Proceedings. AAAI Artificial Intelligence and Interactive Digital Entertainment Conference","volume":"33 1","pages":"292-297"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81310287","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kristen K. Yu, Matthew J. Guzdial, Nathan R Sturtevant, Morgan Cselinacz, Chris Corfe, Izzy Hubert Lyall, Chris Smith
{"title":"Adventures of AI Directors Early in the Development of Nightingale","authors":"Kristen K. Yu, Matthew J. Guzdial, Nathan R Sturtevant, Morgan Cselinacz, Chris Corfe, Izzy Hubert Lyall, Chris Smith","doi":"10.1609/aiide.v18i1.21949","DOIUrl":"https://doi.org/10.1609/aiide.v18i1.21949","url":null,"abstract":"Players can sometimes engage with parts of a video game that they do not enjoy if the game does not try to adapt the experience to the player’s preference. AI directors have been used in the past to tailor player experience to different people. In industry, AI directors are relatively uncommon and are typically domain-specific and rules-based. In this paper, we present a reinforcement learning-based AI director developed for the industry game Nightingale with the help of Inflexion Games. We ran an experiment to evaluate the effectiveness of the AI director in creating a desired player experience, but found inconclusive evidence. In line with this year’s theme, we present our negative results and their implications for future AI directors, along with general discussion from working closely with an industry partner.","PeriodicalId":92576,"journal":{"name":"Proceedings. AAAI Artificial Intelligence and Interactive Digital Entertainment Conference","volume":"41 1","pages":"70-77"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82662729","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Review of Uncertainty for Deep Reinforcement Learning","authors":"Owen Lockwood, Mei Si","doi":"10.48550/arXiv.2208.09052","DOIUrl":"https://doi.org/10.48550/arXiv.2208.09052","url":null,"abstract":"Uncertainty is ubiquitous in games, both in the agents playing games and often in the games themselves. Working with uncertainty is therefore an important component of successful deep reinforcement learning agents. While there has been substantial effort and progress in understanding and working with uncertainty for supervised learning, the body of literature for uncertainty aware deep reinforcement learning is less developed. While many of the same problems regarding uncertainty in neural networks for supervised learning remain for reinforcement learning, there are additional sources of uncertainty due to the nature of an interactable environment. In this work, we provide an overview motivating and presenting existing techniques in uncertainty aware deep reinforcement learning. These works show empirical benefits on a variety of reinforcement learning tasks. This work serves to help to centralize the disparate results and promote future research in this area.","PeriodicalId":92576,"journal":{"name":"Proceedings. AAAI Artificial Intelligence and Interactive Digital Entertainment Conference","volume":"19 1","pages":"155-162"},"PeriodicalIF":0.0,"publicationDate":"2022-08-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85688350","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"tile2tile: Learning Game Filters for Platformer Style Transfer","authors":"Anurag Sarkar, Seth Cooper","doi":"10.48550/arXiv.2208.07699","DOIUrl":"https://doi.org/10.48550/arXiv.2208.07699","url":null,"abstract":"We present tile2tile, an approach for style transfer between levels of tile-based platformer games. Our method involves training models that translate levels from a lower-resolution sketch representation based on tile affordances to the original tile representation for a given game. This enables these models, which we refer to as filters, to translate level sketches into the style of a specific game. Moreover, by converting a level of one game into sketch form and then translating the resulting sketch into the tiles of another game, we obtain a method of style transfer between two games. We use Markov random fields and autoencoders for learning the game filters and apply them to demonstrate style transfer between levels of Super Mario Bros, Kid Icarus, Mega Man and Metroid.","PeriodicalId":92576,"journal":{"name":"Proceedings. AAAI Artificial Intelligence and Interactive Digital Entertainment Conference","volume":"16 1","pages":"53-60"},"PeriodicalIF":0.0,"publicationDate":"2022-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84632879","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lucas N. Ferreira, Lili Mou, Jim Whitehead, Levi H. S. Lelis
{"title":"Controlling Perceived Emotion in Symbolic Music Generation with Monte Carlo Tree Search","authors":"Lucas N. Ferreira, Lili Mou, Jim Whitehead, Levi H. S. Lelis","doi":"10.48550/arXiv.2208.05162","DOIUrl":"https://doi.org/10.48550/arXiv.2208.05162","url":null,"abstract":"This paper presents a new approach for controlling emotion in symbolic music generation with Monte Carlo Tree Search. We use Monte Carlo Tree Search as a decoding mechanism to steer the probability distribution learned by a language model towards a given emotion. At every step of the decoding process, we use Predictor Upper Confidence for Trees (PUCT) to search for sequences that maximize the average values of emotion and quality as given by an emotion classifier and a discriminator, respectively. We use a language model as PUCT's policy and a combination of the emotion classifier and the discriminator as its value function. To decode the next token in a piece of music, we sample from the distribution of node visits created during the search. We evaluate the quality of the generated samples with respect to human-composed pieces using a set of objective metrics computed directly from the generated samples. We also perform a user study to evaluate how human subjects perceive the generated samples' quality and emotion. We compare PUCT against Stochastic Bi-Objective Beam Search (SBBS) and Conditional Sampling (CS). Results suggest that PUCT outperforms SBBS and CS in almost all metrics of music quality and emotion.","PeriodicalId":92576,"journal":{"name":"Proceedings. AAAI Artificial Intelligence and Interactive Digital Entertainment Conference","volume":"577 1","pages":"163-170"},"PeriodicalIF":0.0,"publicationDate":"2022-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77200313","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Hierarchical Dual Attention-Based Recurrent Neural Networks for Individual and Group Activity Recognition in Games","authors":"Sabbir Ahmad, M. S. El-Nasr, Ehsan Elhamifar","doi":"10.1609/aiide.v17i1.18898","DOIUrl":"https://doi.org/10.1609/aiide.v17i1.18898","url":null,"abstract":"We study the problem of simultaneously recognizing complex individual and group activities from spatiotemporal data in games. Recognizing complex player activities is particularly important to understand game dynamics and user behavior having a wide range of applications in game development. To do so, we propose a novel framework by developing a hierarchical dual attention RNN-based method that leverages feature and temporal attention mechanisms in a hierarchical setting for effective discovery of activities using interactions among individuals. We argue that certain activities have dependency on certain features as well as on temporal aspects of the data which can be leveraged by our dual-attention model for recognition. To the best of our knowledge, this work is the first to address activity recognition using spatiotemporal data in games. In addition, we propose using game data as a rich source of obtaining complex group interactions. In this paper, we present two contributions: (1) two annotated game datasets that consist of individual and group activities, (2) our proposed framework improves the state-of-the-art recognition algorithms for spatiotemporal data by experiments on these datasets.","PeriodicalId":92576,"journal":{"name":"Proceedings. AAAI Artificial Intelligence and Interactive Digital Entertainment Conference","volume":"91 1","pages":"116-123"},"PeriodicalIF":0.0,"publicationDate":"2021-10-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76074555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lucas V. S. Pereira, L. Chaimowicz, Levi H. S. Lelis
{"title":"Birds in Boots: Learning to Play Angry Birds with Policy-Guided Search","authors":"Lucas V. S. Pereira, L. Chaimowicz, Levi H. S. Lelis","doi":"10.1609/aiide.v17i1.18893","DOIUrl":"https://doi.org/10.1609/aiide.v17i1.18893","url":null,"abstract":"In this paper we present Birds in Boots (BiB), a system that uses a sampling-based search algorithm to learn a neural policy for solving Angry Birds levels. Our learning procedure is based on the Bootstrap algorithm, which was previously used to learn heuristic functions for solving classic heuristic search problems. BiB starts its learning procedure with a policy given by a randomly initialized neural network. This initial policy is used to guide the search algorithm on a set of procedurally generated Angry Birds levels. The levels the search algorithm is able to solve are used to improve the neural policy. We repeat this procedure a number of times, until solving all levels or reaching a time limit. We perform several experiments with different instances of our method and show that it can solve more levels than other approaches, including learning-based and rule-based methods.","PeriodicalId":92576,"journal":{"name":"Proceedings. AAAI Artificial Intelligence and Interactive Digital Entertainment Conference","volume":"1 1","pages":"74-81"},"PeriodicalIF":0.0,"publicationDate":"2021-10-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85387170","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Tile Embedding: A General Representation for Level Generation","authors":"Mrunal Jadhav, Matthew J. Guzdial","doi":"10.1609/aiide.v17i1.18888","DOIUrl":"https://doi.org/10.1609/aiide.v17i1.18888","url":null,"abstract":"In recent years, Procedural Level Generation via Machine Learning (PLGML) techniques have been applied to generate game levels with machine learning. These approaches rely on human-annotated representations of game levels. Creating annotated datasets for games requires domain knowledge and is time-consuming. Hence, though a large number of video games exist, annotated datasets are curated only for a small handful. Thus current PLGML techniques have been explored in limited domains, with Super Mario Bros. as the most common example. To address this problem, we present tile embeddings, a unified, affordance-rich representation for tile-based 2D games. To learn this embedding, we employ autoencoders trained on the visual and semantic information of tiles from a set of existing, human-annotated games. We evaluate this representation on its ability to predict affordances for unseen tiles, and to serve as a PLGML representation for annotated and unannotated games.","PeriodicalId":92576,"journal":{"name":"Proceedings. AAAI Artificial Intelligence and Interactive Digital Entertainment Conference","volume":"7 1","pages":"34-41"},"PeriodicalIF":0.0,"publicationDate":"2021-10-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83433985","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}