Enhancing AI-Bot Strength and Strategy Diversity in Adversarial Games: A Novel Deep Reinforcement Learning Framework

IF 1.7 4区 计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE
Chenglu Sun;Shuo Shen;Deyi Xue;Wenzhi Tao;Zixia Zhou
{"title":"Enhancing AI-Bot Strength and Strategy Diversity in Adversarial Games: A Novel Deep Reinforcement Learning Framework","authors":"Chenglu Sun;Shuo Shen;Deyi Xue;Wenzhi Tao;Zixia Zhou","doi":"10.1109/TG.2024.3520970","DOIUrl":null,"url":null,"abstract":"Deep reinforcement learning (DRL) has emerged as a leading technique for designing AI-bots in the gaming industry. However, practical implementation of DRL-trained bots often encounter two significant challenges: improving strength and diversifying strategies to satisfy player expectations. We observe that the strength of AI-bots are intrinsically tied to the diversity of emerged strategies. Considering this relationship, we introduce diversity is strength (DIS), a novel DRL training framework capable of concurrently training multiple types of AI-bots for adversarial games. These bots are interconnected through an elaborated history model pool (HMP) structure, thereby improving their strength and strategy diversity to tackle the aforementioned challenges. We further devise a model evaluation and sampling scheme to form the HMP, identify superior models, and enrich the model strategies. The DIS can generate diverse and reliable strategies without the need for human data. This method is validated by achieving first-place finishes in two AI competitions based on complex adversarial games, including Google Research Football and Olympic Games. Experiments demonstrate that bots trained using DIS attain an excellent performance and plentiful strategies. Specifically, diversity analysis demonstrates that the trained bots possess a wealth of strategies, and ablation studies confirm the beneficial impact of the designed modules on the training process.","PeriodicalId":55977,"journal":{"name":"IEEE Transactions on Games","volume":"17 2","pages":"522-535"},"PeriodicalIF":1.7000,"publicationDate":"2024-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Games","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10812583/","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

Abstract

Deep reinforcement learning (DRL) has emerged as a leading technique for designing AI-bots in the gaming industry. However, practical implementation of DRL-trained bots often encounter two significant challenges: improving strength and diversifying strategies to satisfy player expectations. We observe that the strength of AI-bots are intrinsically tied to the diversity of emerged strategies. Considering this relationship, we introduce diversity is strength (DIS), a novel DRL training framework capable of concurrently training multiple types of AI-bots for adversarial games. These bots are interconnected through an elaborated history model pool (HMP) structure, thereby improving their strength and strategy diversity to tackle the aforementioned challenges. We further devise a model evaluation and sampling scheme to form the HMP, identify superior models, and enrich the model strategies. The DIS can generate diverse and reliable strategies without the need for human data. This method is validated by achieving first-place finishes in two AI competitions based on complex adversarial games, including Google Research Football and Olympic Games. Experiments demonstrate that bots trained using DIS attain an excellent performance and plentiful strategies. Specifically, diversity analysis demonstrates that the trained bots possess a wealth of strategies, and ablation studies confirm the beneficial impact of the designed modules on the training process.
增强AI-Bot在对抗游戏中的力量和策略多样性:一种新的深度强化学习框架
深度强化学习(DRL)已成为游戏行业设计ai机器人的领先技术。然而,drl训练的机器人的实际执行通常会遇到两个重大挑战:提高力量和多样化策略以满足玩家的期望。我们观察到,人工智能机器人的力量本质上与新兴策略的多样性有关。考虑到这种关系,我们引入了多样性强度(DIS),这是一种新的DRL训练框架,能够同时训练多种类型的ai机器人进行对抗性游戏。这些机器人通过精心设计的历史模型池(HMP)结构相互连接,从而提高了它们的实力和策略多样性,以应对上述挑战。在此基础上,设计了模型评价和抽样方案,形成了HMP模型,识别出优模型,丰富了模型策略。DIS可以在不需要人工数据的情况下生成多样化和可靠的策略。在谷歌研究足球和奥林匹克运动会等基于复杂对抗性游戏的人工智能比赛中获得第一名,验证了这一方法。实验表明,使用DIS训练的机器人具有优异的性能和丰富的策略。具体而言,多样性分析表明,经过训练的机器人拥有丰富的策略,而消融研究证实了设计模块对训练过程的有益影响。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
IEEE Transactions on Games
IEEE Transactions on Games Engineering-Electrical and Electronic Engineering
CiteScore
4.60
自引率
8.70%
发文量
87
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信