A Scalable Game Theoretic Approach for Coordination of Multiple Dynamic Systems

IF 2.4 Q2 AUTOMATION & CONTROL SYSTEMS
Mostafa M. Shibl;Vijay Gupta
{"title":"A Scalable Game Theoretic Approach for Coordination of Multiple Dynamic Systems","authors":"Mostafa M. Shibl;Vijay Gupta","doi":"10.1109/LCSYS.2024.3501155","DOIUrl":null,"url":null,"abstract":"Learning in games provides a powerful framework to design control policies for self-interested agents that may be coupled through their dynamics, costs, or constraints. We consider the case where the dynamics of the coupled system can be modeled as a Markov potential game. In this case, distributed learning ensures agents’ control policies converge to a Nash equilibrium. However, standard algorithms like natural policy gradient require global state and action knowledge, which does not scale well with more agents. We show that by limiting information flow to local neighborhoods, we can still converge to near-optimal policies. If a game’s global cost function can be decomposed into local costs that align with agent policies at equilibrium, this approach benefits team coordination. We demonstrate this with a sensor coverage problem.","PeriodicalId":37235,"journal":{"name":"IEEE Control Systems Letters","volume":"8 ","pages":"2535-2540"},"PeriodicalIF":2.4000,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Control Systems Letters","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10755096/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

Learning in games provides a powerful framework to design control policies for self-interested agents that may be coupled through their dynamics, costs, or constraints. We consider the case where the dynamics of the coupled system can be modeled as a Markov potential game. In this case, distributed learning ensures agents’ control policies converge to a Nash equilibrium. However, standard algorithms like natural policy gradient require global state and action knowledge, which does not scale well with more agents. We show that by limiting information flow to local neighborhoods, we can still converge to near-optimal policies. If a game’s global cost function can be decomposed into local costs that align with agent policies at equilibrium, this approach benefits team coordination. We demonstrate this with a sensor coverage problem.
多动态系统协调的可扩展博弈论方法
游戏中的学习提供了一个强大的框架,可以通过动态、成本或约束为自利主体设计控制策略。我们考虑耦合系统的动力学可以建模为马尔可夫势对策的情况。在这种情况下,分布式学习确保智能体的控制策略收敛于纳什均衡。然而,像自然策略梯度这样的标准算法需要全局状态和行为知识,这在更多智能体的情况下不能很好地扩展。我们表明,通过限制信息流到当地社区,我们仍然可以收敛到接近最优的政策。如果游戏的全局成本函数可以分解为与代理策略保持平衡的局部成本,那么这种方法就有利于团队协作。我们用一个传感器覆盖问题来证明这一点。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
IEEE Control Systems Letters
IEEE Control Systems Letters Mathematics-Control and Optimization
CiteScore
4.40
自引率
13.30%
发文量
471
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信