Evolutionary multi-agent reinforcement learning in group social dilemmas.

IF 2.7 2区 数学 Q1 MATHEMATICS, APPLIED
Chaos Pub Date : 2025-02-01 DOI:10.1063/5.0246332
B Mintz, F Fu
{"title":"Evolutionary multi-agent reinforcement learning in group social dilemmas.","authors":"B Mintz, F Fu","doi":"10.1063/5.0246332","DOIUrl":null,"url":null,"abstract":"<p><p>Reinforcement learning (RL) is a powerful machine learning technique that has been successfully applied to a wide variety of problems. However, it can be unpredictable and produce suboptimal results in complicated learning environments. This is especially true when multiple agents learn simultaneously, which creates a complex system that is often analytically intractable. Our work considers the fundamental framework of Q-learning in public goods games, where RL individuals must work together to achieve a common goal. This setting allows us to study the tragedy of the commons and free-rider effects in artificial intelligence cooperation, an emerging field with potential to resolve challenging obstacles to the wider application of artificial intelligence. While this social dilemma has been mainly investigated through traditional and evolutionary game theory, our work connects these two approaches by studying agents with an intermediate level of intelligence. We consider the influence of learning parameters on cooperation levels in simulations and a limiting system of differential equations, as well as the effect of evolutionary pressures on exploration rate in both of these models. We find selection for higher and lower levels of exploration, as well as attracting values, and a condition that separates these in a restricted class of games. Our work enhances the theoretical understanding of recent techniques that combine evolutionary algorithms with Q-learning and extends our knowledge of the evolution of machine behavior in social dilemmas.</p>","PeriodicalId":9974,"journal":{"name":"Chaos","volume":"35 2","pages":""},"PeriodicalIF":2.7000,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Chaos","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1063/5.0246332","RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}
引用次数: 0

Abstract

Reinforcement learning (RL) is a powerful machine learning technique that has been successfully applied to a wide variety of problems. However, it can be unpredictable and produce suboptimal results in complicated learning environments. This is especially true when multiple agents learn simultaneously, which creates a complex system that is often analytically intractable. Our work considers the fundamental framework of Q-learning in public goods games, where RL individuals must work together to achieve a common goal. This setting allows us to study the tragedy of the commons and free-rider effects in artificial intelligence cooperation, an emerging field with potential to resolve challenging obstacles to the wider application of artificial intelligence. While this social dilemma has been mainly investigated through traditional and evolutionary game theory, our work connects these two approaches by studying agents with an intermediate level of intelligence. We consider the influence of learning parameters on cooperation levels in simulations and a limiting system of differential equations, as well as the effect of evolutionary pressures on exploration rate in both of these models. We find selection for higher and lower levels of exploration, as well as attracting values, and a condition that separates these in a restricted class of games. Our work enhances the theoretical understanding of recent techniques that combine evolutionary algorithms with Q-learning and extends our knowledge of the evolution of machine behavior in social dilemmas.

群体社会困境中的进化多代理强化学习。
强化学习(RL)是一种强大的机器学习技术,已经成功地应用于各种各样的问题。然而,它可能是不可预测的,并且在复杂的学习环境中产生次优结果。当多个智能体同时学习时尤其如此,这会创建一个通常难以分析的复杂系统。我们的工作考虑了公共产品游戏中Q-learning的基本框架,强化学习个体必须共同努力实现共同目标。这种设置使我们能够研究人工智能合作中的公地悲剧和搭便车效应,这是一个新兴领域,有可能解决人工智能更广泛应用的挑战性障碍。虽然这种社会困境主要通过传统和进化博弈论进行研究,但我们的工作通过研究具有中等智力水平的代理人将这两种方法联系起来。在模拟和微分方程的极限系统中,我们考虑了学习参数对合作水平的影响,以及进化压力对这两种模型中探索速度的影响。我们发现了更高和更低层次探索的选择,以及吸引人的价值,以及在有限的游戏类别中区分这些的条件。我们的工作增强了对将进化算法与q学习相结合的最新技术的理论理解,并扩展了我们对社会困境中机器行为进化的认识。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Chaos
Chaos 物理-物理:数学物理
CiteScore
5.20
自引率
13.80%
发文量
448
审稿时长
2.3 months
期刊介绍: Chaos: An Interdisciplinary Journal of Nonlinear Science is a peer-reviewed journal devoted to increasing the understanding of nonlinear phenomena and describing the manifestations in a manner comprehensible to researchers from a broad spectrum of disciplines.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信