Adaptive Agents and Multi-Agent Systems最新文献

Discovering Consistent Subelections 发现一致的子选择

Adaptive Agents and Multi-Agent Systems Pub Date : 2024-07-26 DOI: 10.5555/3635637.3662948

Lukasz Janeczko, Jérôme Lang, Grzegorz Lisowski, Stanislaw Szufa

引用次数: 0

Strategic Cost Selection in Participatory Budgeting 参与式预算编制中的战略成本选择

Adaptive Agents and Multi-Agent Systems Pub Date : 2024-07-25 DOI: 10.5555/3635637.3663125

Piotr Faliszewski, Lukasz Janeczko, Andrzej Kaczmarczyk, Grzegorz Lisowski, P. Skowron, Stanislaw Szufa

引用次数: 0

Minimizing State Exploration While Searching Graphs with Unknown Obstacles 在搜索具有未知障碍的图时尽量减少状态探索

Adaptive Agents and Multi-Agent Systems Pub Date : 2024-06-01 DOI: 10.5555/3635637.3662959

Daniel Koyfman, Shahaf S. Shperberg, Dor Atzmon, Ariel Felner

{"title":"Minimizing State Exploration While Searching Graphs with Unknown Obstacles","authors":"Daniel Koyfman, Shahaf S. Shperberg, Dor Atzmon, Ariel Felner","doi":"10.5555/3635637.3662959","DOIUrl":"https://doi.org/10.5555/3635637.3662959","url":null,"abstract":"We address the challenge of finding a shortest path in a graph with unknown obstacles where the exploration cost to detect whether a state is free or blocked is very high (e.g., due to sensor activation for obstacle detection). The main objective is to solve the problem while minimizing the number of explorations. To achieve this, we propose MXA∗, a novel heuristic search algorithm based on A∗. The key innovation in MXA∗ lies in modifying the heuristic calculation to avoid obstacles that have already been revealed. Furthermore, this paper makes a noteworthy contribution by introducing the concept of a dynamic heuristic. In contrast to the conventional static heuristic, a dynamic heuristic leverages information that emerges during the search process and adapts its estimations accordingly. By employing a dynamic heuristic, we suggest enhancements to MXA∗ based on real-time information obtained from both the open and closed lists. We demonstrate empirically that MXA∗ finds the shortest path while significantly reducing the number of explored states compared to traditional A∗. The code is available at https: //github.com/bernuly1/MXA-Star.","PeriodicalId":326727,"journal":{"name":"Adaptive Agents and Multi-Agent Systems","volume":"33 3","pages":"1038-1046"},"PeriodicalIF":0.0,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141232947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

vMFER: von Mises-Fisher Experience Resampling Based on Uncertainty of Gradient Directions for Policy Improvement of Actor-Critic Algorithms vMFER：基于梯度方向不确定性的冯-米塞斯-费舍尔经验重采样，用于行动者批判算法的政策改进

Adaptive Agents and Multi-Agent Systems Pub Date : 2024-05-14 DOI: 10.5555/3635637.3663247

Yiwen Zhu, Jinyi Liu, Wenya Wei, Qianyi Fu, Yujing Hu, Zhou Fang, Bo An, Jianye Hao, Tangjie Lv, Changjie Fan

{"title":"vMFER: von Mises-Fisher Experience Resampling Based on Uncertainty of Gradient Directions for Policy Improvement of Actor-Critic Algorithms","authors":"Yiwen Zhu, Jinyi Liu, Wenya Wei, Qianyi Fu, Yujing Hu, Zhou Fang, Bo An, Jianye Hao, Tangjie Lv, Changjie Fan","doi":"10.5555/3635637.3663247","DOIUrl":"https://doi.org/10.5555/3635637.3663247","url":null,"abstract":"Reinforcement Learning (RL) is a widely employed technique in decision-making problems, encompassing two fundamental operations -- policy evaluation and policy improvement. Enhancing learning efficiency remains a key challenge in RL, with many efforts focused on using ensemble critics to boost policy evaluation efficiency. However, when using multiple critics, the actor in the policy improvement process can obtain different gradients. Previous studies have combined these gradients without considering their disagreements. Therefore, optimizing the policy improvement process is crucial to enhance learning efficiency. This study focuses on investigating the impact of gradient disagreements caused by ensemble critics on policy improvement. We introduce the concept of uncertainty of gradient directions as a means to measure the disagreement among gradients utilized in the policy improvement process. Through measuring the disagreement among gradients, we find that transitions with lower uncertainty of gradient directions are more reliable in the policy improvement process. Building on this analysis, we propose a method called von Mises-Fisher Experience Resampling (vMFER), which optimizes the policy improvement process by resampling transitions and assigning higher confidence to transitions with lower uncertainty of gradient directions. Our experiments demonstrate that vMFER significantly outperforms the benchmark and is particularly well-suited for ensemble structures in RL.","PeriodicalId":326727,"journal":{"name":"Adaptive Agents and Multi-Agent Systems","volume":"60 5","pages":"2621-2623"},"PeriodicalIF":0.0,"publicationDate":"2024-05-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140978793","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Reinforcement Nash Equilibrium Solver 强化纳什均衡求解器

Adaptive Agents and Multi-Agent Systems Pub Date : 2024-05-06 DOI: 10.5555/3635637.3663224

Xinrun Wang, Chang Yang, Shuxin Li, Pengdeng Li, Xiao Huang, Hau Chan, Bo An

{"title":"Reinforcement Nash Equilibrium Solver","authors":"Xinrun Wang, Chang Yang, Shuxin Li, Pengdeng Li, Xiao Huang, Hau Chan, Bo An","doi":"10.5555/3635637.3663224","DOIUrl":"https://doi.org/10.5555/3635637.3663224","url":null,"abstract":"Nash Equilibrium (NE) is the canonical solution concept of game theory, which provides an elegant tool to understand the rationalities. Though mixed strategy NE exists in any game with finite players and actions, computing NE in two- or multi-player general-sum games is PPAD-Complete. Various alternative solutions, e.g., Correlated Equilibrium (CE), and learning methods, e.g., fictitious play (FP), are proposed to approximate NE. For convenience, we call these methods as\"inexact solvers\", or\"solvers\"for short. However, the alternative solutions differ from NE and the learning methods generally fail to converge to NE. Therefore, in this work, we propose REinforcement Nash Equilibrium Solver (RENES), which trains a single policy to modify the games with different sizes and applies the solvers on the modified games where the obtained solution is evaluated on the original games. Specifically, our contributions are threefold. i) We represent the games as $alpha$-rank response graphs and leverage graph neural network (GNN) to handle the games with different sizes as inputs; ii) We use tensor decomposition, e.g., canonical polyadic (CP), to make the dimension of modifying actions fixed for games with different sizes; iii) We train the modifying strategy for games with the widely-used proximal policy optimization (PPO) and apply the solvers to solve the modified games, where the obtained solution is evaluated on original games. Extensive experiments on large-scale normal-form games show that our method can further improve the approximation of NE of different solvers, i.e., $alpha$-rank, CE, FP and PRD, and can be generalized to unseen games.","PeriodicalId":326727,"journal":{"name":"Adaptive Agents and Multi-Agent Systems","volume":"57 35","pages":"2552-2554"},"PeriodicalIF":0.0,"publicationDate":"2024-05-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141009450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

On the Utility of External Agent Intention Predictor for Human-AI Coordination 论外部代理意图预测器对人机协调的效用

Adaptive Agents and Multi-Agent Systems Pub Date : 2024-05-03 DOI: 10.5555/3635637.3663222

Chenxu Wang, Zilong Chen, Huaping Liu

{"title":"On the Utility of External Agent Intention Predictor for Human-AI Coordination","authors":"Chenxu Wang, Zilong Chen, Huaping Liu","doi":"10.5555/3635637.3663222","DOIUrl":"https://doi.org/10.5555/3635637.3663222","url":null,"abstract":"Reaching a consensus on the team plans is vital to human-AI coordination. Although previous studies provide approaches through communications in various ways, it could still be hard to coordinate when the AI has no explainable plan to communicate. To cover this gap, we suggest incorporating external models to assist humans in understanding the intentions of AI agents. In this paper, we propose a two-stage paradigm that first trains a Theory of Mind (ToM) model from collected offline trajectories of the target agent, and utilizes the model in the process of human-AI collaboration by real-timely displaying the future action predictions of the target agent. Such a paradigm leaves the AI agent as a black box and thus is available for improving any agents. To test our paradigm, we further implement a transformer-based predictor as the ToM model and develop an extended online human-AI collaboration platform for experiments. The comprehensive experimental results verify that human-AI teams can achieve better performance with the help of our model. A user assessment attached to the experiment further demonstrates that our paradigm can significantly enhance the situational awareness of humans. Our study presents the potential to augment the ability of humans via external assistance in human-AI collaboration, which may further inspire future research.","PeriodicalId":326727,"journal":{"name":"Adaptive Agents and Multi-Agent Systems","volume":"54 6","pages":"2546-2548"},"PeriodicalIF":0.0,"publicationDate":"2024-05-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141016713","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MESA: Cooperative Meta-Exploration in Multi-Agent Learning through Exploiting State-Action Space Structure MESA：通过利用状态-行动空间结构在多代理学习中进行合作元探索

Adaptive Agents and Multi-Agent Systems Pub Date : 2024-05-01 DOI: 10.5555/3635637.3663073

Zhicheng Zhang, Yancheng Liang, Yi Wu, Fei Fang

引用次数: 1

Dual Role AoI-based Incentive Mechanism for HD map Crowdsourcing 基于双角色 AoI 的高清地图众包激励机制

Adaptive Agents and Multi-Agent Systems Pub Date : 2024-05-01 DOI: 10.5555/3635637.3663230

Wentao Ye, Bo Liu, Yuan Luo, Jianwei Huang

引用次数: 0

STV+Reductions: Towards Practical Verification of Strategic Ability Using Model Reductions STV+约简:利用模型约简实现战略能力的实际验证

Adaptive Agents and Multi-Agent Systems Pub Date : 2023-10-27 DOI: 10.5555/3463952.3464232

Damian Kurpiewski, Witold Pazderski, W. Jamroga, Yan Kim

引用次数: 12

Formally-Sharp DAgger for MCTS: Lower-Latency Monte Carlo Tree Search using Data Aggregation with Formal Methods MCTS的形式尖锐匕首:使用数据聚合和形式方法的低延迟蒙特卡罗树搜索

Adaptive Agents and Multi-Agent Systems Pub Date : 2023-08-15 DOI: 10.5555/3545946.3598783

D. Chakraborty, Damien Busatto-Gaston, Jean-François Raskin, G. Pérez

引用次数: 1