在潜在均值场博弈中学习最优策略：平滑政策迭代算法

IF 2.2 2区数学 Q2 AUTOMATION & CONTROL SYSTEMS

SIAM Journal on Control and Optimization Pub Date : 2024-01-24 DOI:10.1137/22m1539861

Qing Tang, Jiahao Song

{"title":"在潜在均值场博弈中学习最优策略：平滑政策迭代算法","authors":"Qing Tang, Jiahao Song","doi":"10.1137/22m1539861","DOIUrl":null,"url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 1, Page 351-375, February 2024. <br/> Abstract. We introduce two smoothed policy iteration algorithms (SPIs) as rules for learning policies and methods for computing Nash equilibria in second order potential mean field games (MFGs). Global convergence is proved if the coupling term in the MFG system satisfies the Lasry–Lions monotonicity condition. Local convergence to a stable solution is proved for a system which may have multiple solutions. The convergence analysis shows close connections between SPIs and the fictitious play algorithm, which has been widely studied in the MFG literature. Numerical simulation results based on finite difference schemes are presented to supplement the theoretical analysis.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2000,"publicationDate":"2024-01-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Learning Optimal Policies in Potential Mean Field Games: Smoothed Policy Iteration Algorithms\",\"authors\":\"Qing Tang, Jiahao Song\",\"doi\":\"10.1137/22m1539861\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"SIAM Journal on Control and Optimization, Volume 62, Issue 1, Page 351-375, February 2024. <br/> Abstract. We introduce two smoothed policy iteration algorithms (SPIs) as rules for learning policies and methods for computing Nash equilibria in second order potential mean field games (MFGs). Global convergence is proved if the coupling term in the MFG system satisfies the Lasry–Lions monotonicity condition. Local convergence to a stable solution is proved for a system which may have multiple solutions. The convergence analysis shows close connections between SPIs and the fictitious play algorithm, which has been widely studied in the MFG literature. Numerical simulation results based on finite difference schemes are presented to supplement the theoretical analysis.\",\"PeriodicalId\":49531,\"journal\":{\"name\":\"SIAM Journal on Control and Optimization\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.2000,\"publicationDate\":\"2024-01-24\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"SIAM Journal on Control and Optimization\",\"FirstCategoryId\":\"100\",\"ListUrlMain\":\"https://doi.org/10.1137/22m1539861\",\"RegionNum\":2,\"RegionCategory\":\"数学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"AUTOMATION & CONTROL SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"SIAM Journal on Control and Optimization","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1137/22m1539861","RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}

引用次数: 0

摘要

SIAM 控制与优化期刊》第 62 卷第 1 期第 351-375 页，2024 年 2 月。摘要。我们介绍了两种平滑策略迭代算法（SPIs），作为二阶势均场博弈（MFGs）中学习策略的规则和计算纳什均衡的方法。如果 MFG 系统中的耦合项满足 Lasry-Lions 单调性条件，就能证明全局收敛性。对于可能有多个解的系统，证明了向稳定解的局部收敛。收敛性分析表明了 SPI 与虚构游戏算法之间的密切联系，后者已在 MFG 文献中得到广泛研究。本文给出了基于有限差分方案的数值模拟结果，以补充理论分析。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Learning Optimal Policies in Potential Mean Field Games: Smoothed Policy Iteration Algorithms

SIAM Journal on Control and Optimization, Volume 62, Issue 1, Page 351-375, February 2024.
Abstract. We introduce two smoothed policy iteration algorithms (SPIs) as rules for learning policies and methods for computing Nash equilibria in second order potential mean field games (MFGs). Global convergence is proved if the coupling term in the MFG system satisfies the Lasry–Lions monotonicity condition. Local convergence to a stable solution is proved for a system which may have multiple solutions. The convergence analysis shows close connections between SPIs and the fictitious play algorithm, which has been widely studied in the MFG literature. Numerical simulation results based on finite difference schemes are presented to supplement the theoretical analysis.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

SIAM Journal on Control and Optimization 数学-应用数学

CiteScore

4.00

自引率

4.50%

发文量

143

审稿时长

12 months

期刊介绍： SIAM Journal on Control and Optimization (SICON) publishes original research articles on the mathematics and applications of control theory and certain parts of optimization theory. Papers considered for publication must be significant at both the mathematical level and the level of applications or potential applications. Papers containing mostly routine mathematics or those with no discernible connection to control and systems theory or optimization will not be considered for publication. From time to time, the journal will also publish authoritative surveys of important subject areas in control theory and optimization whose level of maturity permits a clear and unified exposition. The broad areas mentioned above are intended to encompass a wide range of mathematical techniques and scientific, engineering, economic, and industrial applications. These include stochastic and deterministic methods in control, estimation, and identification of systems; modeling and realization of complex control systems; the numerical analysis and related computational methodology of control processes and allied issues; and the development of mathematical theories and techniques that give new insights into old problems or provide the basis for further progress in control theory and optimization. Within the field of optimization, the journal focuses on the parts that are relevant to dynamic and control systems. Contributions to numerical methodology are also welcome in accordance with these aims, especially as related to large-scale problems and decomposition as well as to fundamental questions of convergence and approximation.