arXiv - CS - Multiagent Systems最新文献_第10页

Finite-Time Analysis of Asynchronous Multi-Agent TD Learning 异步多代理 TD 学习的有限时间分析

arXiv - CS - Multiagent Systems Pub Date : 2024-07-29 DOI: arxiv-2407.20441

Nicolò Dal Fabbro, Arman Adibi, Aritra Mitra, George J. Pappas

{"title":"Finite-Time Analysis of Asynchronous Multi-Agent TD Learning","authors":"Nicolò Dal Fabbro, Arman Adibi, Aritra Mitra, George J. Pappas","doi":"arxiv-2407.20441","DOIUrl":"https://doi.org/arxiv-2407.20441","url":null,"abstract":"Recent research endeavours have theoretically shown the beneficial effect of\u0000cooperation in multi-agent reinforcement learning (MARL). In a setting\u0000involving $N$ agents, this beneficial effect usually comes in the form of an\u0000$N$-fold linear convergence speedup, i.e., a reduction - proportional to $N$ -\u0000in the number of iterations required to reach a certain convergence precision.\u0000In this paper, we show for the first time that this speedup property also holds\u0000for a MARL framework subject to asynchronous delays in the local agents'\u0000updates. In particular, we consider a policy evaluation problem in which\u0000multiple agents cooperate to evaluate a common policy by communicating with a\u0000central aggregator. In this setting, we study the finite-time convergence of\u0000texttt{AsyncMATD}, an asynchronous multi-agent temporal difference (TD)\u0000learning algorithm in which agents' local TD update directions are subject to\u0000asynchronous bounded delays. Our main contribution is providing a finite-time\u0000analysis of texttt{AsyncMATD}, for which we establish a linear convergence\u0000speedup while highlighting the effect of time-varying asynchronous delays on\u0000the resulting convergence rate.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141871368","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Quantum Computing and Neuromorphic Computing for Safe, Reliable, and explainable Multi-Agent Reinforcement Learning: Optimal Control in Autonomous Robotics 量子计算和神经形态计算用于安全、可靠和可解释的多代理强化学习：自主机器人技术中的最优控制

arXiv - CS - Multiagent Systems Pub Date : 2024-07-29 DOI: arxiv-2408.03884

Mazyar Taghavi

{"title":"Quantum Computing and Neuromorphic Computing for Safe, Reliable, and explainable Multi-Agent Reinforcement Learning: Optimal Control in Autonomous Robotics","authors":"Mazyar Taghavi","doi":"arxiv-2408.03884","DOIUrl":"https://doi.org/arxiv-2408.03884","url":null,"abstract":"This paper investigates the utilization of Quantum Computing and Neuromorphic\u0000Computing for Safe, Reliable, and Explainable Multi_Agent Reinforcement\u0000Learning (MARL) in the context of optimal control in autonomous robotics. The\u0000objective was to address the challenges of optimizing the behavior of\u0000autonomous agents while ensuring safety, reliability, and explainability.\u0000Quantum Computing techniques, including Quantum Approximate Optimization\u0000Algorithm (QAOA), were employed to efficiently explore large solution spaces\u0000and find approximate solutions to complex MARL problems. Neuromorphic\u0000Computing, inspired by the architecture of the human brain, provided parallel\u0000and distributed processing capabilities, which were leveraged to develop\u0000intelligent and adaptive systems. The combination of these technologies held\u0000the potential to enhance the safety, reliability, and explainability of MARL in\u0000autonomous robotics. This research contributed to the advancement of autonomous\u0000robotics by exploring cutting-edge technologies and their applications in\u0000multi-agent systems. Codes and data are available.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141949342","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Eliminating Majority Illusion is Easy 消除多数幻觉很容易

arXiv - CS - Multiagent Systems Pub Date : 2024-07-29 DOI: arxiv-2407.20187

Jack Dippel, Max Dupré la Tour, April Niu, Sanjukta Roy, Adrian Vetta

引用次数: 0

Navigation services amplify concentration of traffic and emissions in our cities 导航服务扩大了城市交通和排放的集中度

arXiv - CS - Multiagent Systems Pub Date : 2024-07-29 DOI: arxiv-2407.20004

Giuliano Cornacchia, Mirco Nanni, Dino Pedreschi, Luca Pappalardo

{"title":"Navigation services amplify concentration of traffic and emissions in our cities","authors":"Giuliano Cornacchia, Mirco Nanni, Dino Pedreschi, Luca Pappalardo","doi":"arxiv-2407.20004","DOIUrl":"https://doi.org/arxiv-2407.20004","url":null,"abstract":"The proliferation of human-AI ecosystems involving human interaction with\u0000algorithms, such as assistants and recommenders, raises concerns about\u0000large-scale social behaviour. Despite evidence of such phenomena across several\u0000contexts, the collective impact of GPS navigation services remains unclear:\u0000while beneficial to the user, they can also cause chaos if too many vehicles\u0000are driven through the same few roads. Our study employs a simulation framework\u0000to assess navigation services' influence on road network usage and CO2\u0000emissions. The results demonstrate a universal pattern of amplified conformity:\u0000increasing adoption rates of navigation services cause a reduction of route\u0000diversity of mobile travellers and increased concentration of traffic and\u0000emissions on fewer roads, thus exacerbating an unequal distribution of negative\u0000externalities on selected neighbourhoods. Although navigation services\u0000recommendations can help reduce CO2 emissions when their adoption rate is low,\u0000these benefits diminish or even disappear when the adoption rate is high and\u0000exceeds a certain city- and service-dependent threshold. We summarize these\u0000discoveries in a non-linear function that connects the marginal increase of\u0000conformity with the marginal reduction in CO2 emissions. Our simulation\u0000approach addresses the challenges posed by the complexity of transportation\u0000systems and the lack of data and algorithmic transparency.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141871370","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Mechanism Design for Locating Facilities with Capacities with Insufficient Resources 在资源不足的情况下为有能力的设施选址的机制设计

arXiv - CS - Multiagent Systems Pub Date : 2024-07-26 DOI: arxiv-2407.18547

Gennaro Auricchio, Harry J. Clough, Jie Zhang

{"title":"Mechanism Design for Locating Facilities with Capacities with Insufficient Resources","authors":"Gennaro Auricchio, Harry J. Clough, Jie Zhang","doi":"arxiv-2407.18547","DOIUrl":"https://doi.org/arxiv-2407.18547","url":null,"abstract":"This paper explores the Mechanism Design aspects of the $m$-Capacitated\u0000Facility Location Problem where the total facility capacity is less than the\u0000number of agents. Following the framework outlined by Aziz et al., the Social\u0000Welfare of the facility location is determined through a\u0000First-Come-First-Served (FCFS) game, in which agents compete once the facility\u0000positions are established. When the number of facilities is $m > 1$, the Nash\u0000Equilibrium (NE) of the FCFS game is not unique, making the utility of the\u0000agents and the concept of truthfulness unclear. To tackle these issues, we\u0000consider absolutely truthful mechanisms, i.e. mechanisms that prevent agents\u0000from misreporting regardless of the strategies used during the FCFS game. We\u0000combine this stricter truthfulness requirement with the notion of Equilibrium\u0000Stable (ES) mechanisms, which are mechanisms whose Social Welfare does not\u0000depend on the NE of the FCFS game. We demonstrate that the class of percentile\u0000mechanisms is absolutely truthful and identify the conditions under which they\u0000are ES. We also show that the approximation ratio of each ES percentile\u0000mechanism is bounded and determine its value. Notably, when all the facilities\u0000have the same capacity and the number of agents is sufficiently large, it is\u0000possible to achieve an approximation ratio smaller than $1+frac{1}{2m-1}$.\u0000Finally, we extend our study to encompass higher-dimensional problems. Within\u0000this framework, we demonstrate that the class of ES percentile mechanisms is\u0000even more restricted and characterize the mechanisms that are both ES and\u0000absolutely truthful. We further support our findings by empirically evaluating\u0000the performance of the mechanisms when the agents are the samples of a\u0000distribution.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141871371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Socially efficient mechanism on the minimum budget 最低预算的社会效率机制

arXiv - CS - Multiagent Systems Pub Date : 2024-07-26 DOI: arxiv-2407.18515

Hirota Kinoshita, Takayuki Osogami, Kohei Miyaguchi

引用次数: 0

Principal-Agent Reinforcement Learning 主代理强化学习

arXiv - CS - Multiagent Systems Pub Date : 2024-07-25 DOI: arxiv-2407.18074

Dima Ivanov, Paul Dütting, Inbal Talgam-Cohen, Tonghan Wang, David C. Parkes

引用次数: 0

Very Large-Scale Multi-Agent Simulation in AgentScope AgentScope 中的超大规模多代理模拟

arXiv - CS - Multiagent Systems Pub Date : 2024-07-25 DOI: arxiv-2407.17789

Xuchen Pan, Dawei Gao, Yuexiang Xie, Zhewei Wei, Yaliang Li, Bolin Ding, Ji-Rong Wen, Jingren Zhou

{"title":"Very Large-Scale Multi-Agent Simulation in AgentScope","authors":"Xuchen Pan, Dawei Gao, Yuexiang Xie, Zhewei Wei, Yaliang Li, Bolin Ding, Ji-Rong Wen, Jingren Zhou","doi":"arxiv-2407.17789","DOIUrl":"https://doi.org/arxiv-2407.17789","url":null,"abstract":"Recent advances in large language models (LLMs) have opened new avenues for\u0000applying multi-agent systems in very large-scale simulations. However, there\u0000remain several challenges when conducting multi-agent simulations with existing\u0000platforms, such as limited scalability and low efficiency, unsatisfied agent\u0000diversity, and effort-intensive management processes. To address these\u0000challenges, we develop several new features and components for AgentScope, a\u0000user-friendly multi-agent platform, enhancing its convenience and flexibility\u0000for supporting very large-scale multi-agent simulations. Specifically, we\u0000propose an actor-based distributed mechanism as the underlying technological\u0000infrastructure towards great scalability and high efficiency, and provide\u0000flexible environment support for simulating various real-world scenarios, which\u0000enables parallel execution of multiple agents, centralized workflow\u0000orchestration, and both inter-agent and agent-environment interactions among\u0000agents. Moreover, we integrate an easy-to-use configurable tool and an\u0000automatic background generation pipeline in AgentScope, simplifying the process\u0000of creating agents with diverse yet detailed background settings. Last but not\u0000least, we provide a web-based interface for conveniently monitoring and\u0000managing a large number of agents that might deploy across multiple devices. We\u0000conduct a comprehensive simulation to demonstrate the effectiveness of the\u0000proposed enhancements in AgentScope, and provide detailed observations and\u0000discussions to highlight the great potential of applying multi-agent systems in\u0000large-scale simulations. The source code is released on GitHub at\u0000https://github.com/modelscope/agentscope to inspire further research and\u0000development in large-scale multi-agent simulations.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141777169","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Strategic Cost Selection in Participatory Budgeting 参与式预算编制中的战略成本选择

arXiv - CS - Multiagent Systems Pub Date : 2024-07-25 DOI: arxiv-2407.18092

Piotr Faliszewski, Łukasz Janeczko, Andrzej Kaczmarczyk, Grzegorz Lisowski, Piotr Skowron, Stanisław Szufa

引用次数: 0

Stochastic Games with Minimally Bounded Action Costs 行动成本最小有界的随机博弈

arXiv - CS - Multiagent Systems Pub Date : 2024-07-25 DOI: arxiv-2407.18010

David Mguni

{"title":"Stochastic Games with Minimally Bounded Action Costs","authors":"David Mguni","doi":"arxiv-2407.18010","DOIUrl":"https://doi.org/arxiv-2407.18010","url":null,"abstract":"In many multi-player interactions, players incur strictly positive costs each\u0000time they execute actions e.g. 'menu costs' or transaction costs in financial\u0000systems. Since acting at each available opportunity would accumulate\u0000prohibitively large costs, the resulting decision problem is one in which\u0000players must make strategic decisions about when to execute actions in addition\u0000to their choice of action. This paper analyses a discrete-time stochastic game\u0000(SG) in which players face minimally bounded positive costs for each action and\u0000influence the system using impulse controls. We prove SGs of two-sided impulse\u0000control have a unique value and characterise the saddle point equilibrium in\u0000which the players execute actions at strategically chosen times in accordance\u0000with Markovian strategies. We prove the game respects a dynamic programming\u0000principle and that the Markov perfect equilibrium can be computed as a limit\u0000point of a sequence of Bellman operations. We then introduce a new Q-learning\u0000variant which we show converges almost surely to the value of the game enabling\u0000solutions to be extracted in unknown settings. Lastly, we extend our results to\u0000settings with budgetory constraints.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141777168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0