Autonomous Agents and Multi-Agent Systems最新文献_第8页

Effect of asynchronous execution and imperfect communication on max-sum belief propagation 异步执行和不完全通信对最大和置信传播的影响

IF 1.9 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2023-09-14 DOI: 10.1007/s10458-023-09621-w

Roie Zivan, Ben Rachmut, Omer Perry, William Yeoh

{"title":"Effect of asynchronous execution and imperfect communication on max-sum belief propagation","authors":"Roie Zivan, Ben Rachmut, Omer Perry, William Yeoh","doi":"10.1007/s10458-023-09621-w","DOIUrl":"10.1007/s10458-023-09621-w","url":null,"abstract":"<div><p>Max-sum is a version of belief propagation that was adapted for solving distributed constraint optimization problems. It has been studied theoretically and empirically, extended to versions that improve solution quality and converge rapidly, and is applicable to multiple distributed applications. The algorithm was presented both as synchronous and asynchronous algorithms. However, neither the differences in the performance of the two execution versions nor the implications of imperfect communication (i.e., massage delay and message loss) on the two versions have been investigated to the best of our knowledge. We contribute to the body of knowledge on Max-sum by: (1) Establishing the theoretical differences between the two execution versions of the algorithm, focusing on the construction of beliefs; (2) Empirically evaluating the differences between the solutions generated by the two versions of the algorithm, with and without message delay or loss; and (3) Establishing both theoretically and empirically the positive effect of damping on reducing the differences between the two versions. Our results indicate that, in contrast to recent published results indicating that message latency has a drastic (positive) effect on the performance of distributed local search algorithms, the effect of imperfect communication on Damped Max-sum (DMS) is minor. The version of Max-sum that includes both damping and splitting of function nodes converges to high quality solutions very fast, even when a large percentage of the messages sent by agents do not arrive at their destinations. Moreover, the quality of solutions in the different versions of DMS is dependent of the number of messages that were received by the agents, regardless of the amount of time they were delayed or if these messages are only a portion of the total number of messages that was sent by the agents.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"37 2","pages":""},"PeriodicalIF":1.9,"publicationDate":"2023-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"50482745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Fairness criteria for allocating indivisible chores: connections and efficiencies 分配不可分割家务的公平标准：联系和效率

IF 1.9 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2023-08-31 DOI: 10.1007/s10458-023-09618-5

Ankang Sun, Bo Chen, Xuan Vinh Doan

引用次数: 0

A multi-scenario approach to continuously learn and understand norm violations 持续学习和理解违反规范的多场景方法

IF 1.9 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2023-08-16 DOI: 10.1007/s10458-023-09619-4

Thiago Freitas dos Santos, Nardine Osman, Marco Schorlemmer

{"title":"A multi-scenario approach to continuously learn and understand norm violations","authors":"Thiago Freitas dos Santos, Nardine Osman, Marco Schorlemmer","doi":"10.1007/s10458-023-09619-4","DOIUrl":"10.1007/s10458-023-09619-4","url":null,"abstract":"<div><p>Using norms to guide and coordinate interactions has gained tremendous attention in the multiagent community. However, new challenges arise as the interest moves towards dynamic socio-technical systems, where human and software agents interact, and interactions are required to adapt to changing human needs. For instance, different agents (human or software) might not have the same understanding of what it means to violate a norm (e.g., what characterizes hate speech), or their understanding of a norm might change over time (e.g., what constitutes an acceptable response time). The challenge is to address these issues by learning to detect norm violations from the limited interaction data and to explain the reasons for such violations. To do that, we propose a framework that combines Machine Learning (ML) models and incremental learning techniques. Our proposal is equipped to solve tasks in both tabular and text classification scenarios. Incremental learning is used to continuously update the base ML models as interactions unfold, ensemble learning is used to handle the imbalance class distribution of the interaction stream, Pre-trained Language Model (PLM) is used to learn from text sentences, and Integrated Gradients (IG) is the interpretability algorithm. We evaluate the proposed approach in the use case of Wikipedia article edits, where interactions revolve around editing articles, and the norm in question is prohibiting vandalism. Results show that the proposed framework can learn to detect norm violation in a setting with data imbalance and concept drift.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"37 2","pages":""},"PeriodicalIF":1.9,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10458-023-09619-4.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41757932","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Hardness of candidate nomination 候选人提名的难度

IF 1.9 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2023-08-16 DOI: 10.1007/s10458-023-09622-9

Katarína Cechlárová, Julien Lesca, Diana Trellová, Martina Hančová, Jozef Hanč

引用次数: 0

Classifying ambiguous identities in hidden-role Stochastic games with multi-agent reinforcement learning 基于多智能体强化学习的隐藏角色随机博弈模糊身份分类

IF 1.9 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2023-08-11 DOI: 10.1007/s10458-023-09620-x

Shijie Han, Siyuan Li, Bo An, Wei Zhao, Peng Liu

{"title":"Classifying ambiguous identities in hidden-role Stochastic games with multi-agent reinforcement learning","authors":"Shijie Han, Siyuan Li, Bo An, Wei Zhao, Peng Liu","doi":"10.1007/s10458-023-09620-x","DOIUrl":"10.1007/s10458-023-09620-x","url":null,"abstract":"<div><p>Multi-agent reinforcement learning (MARL) is a prevalent learning paradigm for solving stochastic games. In most MARL studies, agents in a game are defined as teammates or enemies beforehand, and the relationships among the agents (i.e., their <i>identities</i>) remain fixed throughout the game. However, in real-world problems, the agent relationships are commonly unknown in advance or dynamically changing. Many multi-party interactions start off by asking: who is on my team? This question arises whether it is the first day at the stock exchange or the kindergarten. Therefore, training policies for such situations in the face of imperfect information and ambiguous <i>identities</i> is an important problem that needs to be addressed. In this work, we develop a novel identity detection reinforcement learning (IDRL) framework that allows an agent to dynamically infer the identities of nearby agents and select an appropriate policy to accomplish the task. In the IDRL framework, a relation network is constructed to deduce the identities of other agents by observing the behaviors of the agents. A danger network is optimized to estimate the risk of false-positive identifications. Beyond that, we propose an intrinsic reward that balances the need to maximize external rewards and accurate identification. After identifying the cooperation-competition pattern among the agents, IDRL applies one of the off-the-shelf MARL methods to learn the policy. To evaluate the proposed method, we conduct experiments on <i>Red-10</i> card-shedding game, and the results show that IDRL achieves superior performance over other state-of-the-art MARL methods. Impressively, the relation network has the par performance to identify the identities of agents with top human players; the danger network reasonably avoids the risk of imperfect identification. The code to reproduce all the reported results is available online at https://github.com/MR-BENjie/IDRL.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"37 2","pages":""},"PeriodicalIF":1.9,"publicationDate":"2023-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43149265","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Combining theory of mind and abductive reasoning in agent-oriented programming 面向agent程序设计中思维理论与溯因推理的结合

IF 1.9 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2023-08-11 DOI: 10.1007/s10458-023-09613-w

Nieves Montes, Michael Luck, Nardine Osman, Odinaldo Rodrigues, Carles Sierra

{"title":"Combining theory of mind and abductive reasoning in agent-oriented programming","authors":"Nieves Montes, Michael Luck, Nardine Osman, Odinaldo Rodrigues, Carles Sierra","doi":"10.1007/s10458-023-09613-w","DOIUrl":"10.1007/s10458-023-09613-w","url":null,"abstract":"<div><p>This paper presents a novel model, called T<span>om</span>A<span>bd</span>, that endows autonomous agents with Theory of Mind capabilities. T<span>om</span>A<span>bd</span> agents are able to simulate the perspective of the world that their peers have and reason from their perspective. Furthermore, T<span>om</span>A<span>bd</span> agents can reason from the perspective of others down to an <i>arbitrary level of recursion</i>, using Theory of Mind of <span>(n^{text {th}})</span> order. By combining the previous capability with abductive reasoning, T<span>om</span>A<span>bd</span> agents can infer the beliefs that others were relying upon to select their actions, hence putting them in a more informed position when it comes to their own decision-making. We have tested the T<span>om</span>A<span>bd</span> model in the challenging domain of Hanabi, a game characterised by cooperation and imperfect information. Our results show that the abilities granted by the T<span>om</span>A<span>bd</span> model boost the performance of the team along a variety of metrics, including final score, efficiency of communication, and uncertainty reduction.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"37 2","pages":""},"PeriodicalIF":1.9,"publicationDate":"2023-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10458-023-09613-w.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47381284","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A novel policy-graph approach with natural language and counterfactual abstractions for explaining reinforcement learning agents 用自然语言和反事实抽象的策略图方法解释强化学习代理

IF 1.9 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2023-08-09 DOI: 10.1007/s10458-023-09615-8

Tongtong Liu, Joe McCalmon, Thai Le, Md Asifur Rahman, Dongwon Lee, Sarra Alqahtani

{"title":"A novel policy-graph approach with natural language and counterfactual abstractions for explaining reinforcement learning agents","authors":"Tongtong Liu, Joe McCalmon, Thai Le, Md Asifur Rahman, Dongwon Lee, Sarra Alqahtani","doi":"10.1007/s10458-023-09615-8","DOIUrl":"10.1007/s10458-023-09615-8","url":null,"abstract":"<div><p>As reinforcement learning (RL) continues to improve and be applied in situations alongside humans, the need to explain the learned behaviors of RL agents to end-users becomes more important. Strategies for explaining the reasoning behind an agent’s policy, called <i>policy-level explanations</i>, can lead to important insights about both the task and the agent’s behaviors. Following this line of research, in this work, we propose a novel approach, named as <span>CAPS</span>, that summarizes an agent’s policy in the form of a directed graph with natural language descriptions. A decision tree based clustering method is utilized to abstract the state space of the task into fewer, condensed states which makes the policy graphs more digestible to end-users. We then use the user-defined predicates to enrich the abstract states with semantic meaning. To introduce counterfactual state explanations to the policy graph, we first identify the critical states in the graph then develop a novel counterfactual explanation method based on action perturbation in those critical states. We generate explanation graphs using <span>CAPS</span> on 5 RL tasks, using both deterministic and stochastic policies. We also evaluate the effectiveness of CAPS on human participants who are not RL experts in two user studies. When provided with our explanation graph, end-users are able to accurately interpret policies of trained RL agents 80% of the time, compared to 10% when provided with the next best baseline and <span>(68.2%)</span> of users demonstrated an increase in their confidence in understanding an agent’s behavior after provided with the counterfactual explanations.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"37 2","pages":""},"PeriodicalIF":1.9,"publicationDate":"2023-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46086354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Full communication memory networks for team-level cooperation learning 团队级合作学习的全沟通记忆网络

IF 1.9 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2023-08-07 DOI: 10.1007/s10458-023-09617-6

Yutong Wang, Yizhuo Wang, Guillaume Sartoretti

{"title":"Full communication memory networks for team-level cooperation learning","authors":"Yutong Wang, Yizhuo Wang, Guillaume Sartoretti","doi":"10.1007/s10458-023-09617-6","DOIUrl":"10.1007/s10458-023-09617-6","url":null,"abstract":"<div><p>Communication in multi-agent systems is a key driver of team-level cooperation, for instance allowing individual agents to augment their knowledge about the world in partially-observable environments. In this paper, we propose two reinforcement learning-based multi-agent models, namely FCMNet and FCMTran. The two models both allow agents to simultaneously learn a differentiable communication mechanism that connects all agents as well as a common, cooperative policy conditioned upon received information. FCMNet utilizes multiple directional Long Short-Term Memory chains to sequentially transmit and encode the current observation-based messages sent by every other agent at each timestep. FCMTran further relies on the encoder of a modified transformer to simultaneously aggregate multiple self-generated messages sent by all agents at the previous timestep into a single message that is used in the current timestep. Results from evaluating our models on a challenging set of StarCraft II micromanagement tasks with shared rewards show that FCMNet and FCMTran both outperform recent communication-based methods and value decomposition methods in almost all tested StarCraft II micromanagement tasks. We further improve the performance of our models by combining them with value decomposition techniques; there, in particular, we show that FCMTran with value decomposition significantly pushes the state-of-the-art on one of the hardest benchmark tasks without any task-specific tuning. We also investigate the robustness of FCMNet under communication disturbances (i.e., binarized messages, random message loss, and random communication order) in an asymmetric collaborative pathfinding task with individual rewards, demonstrating FMCNet’s potential applicability in real-world robotic tasks.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"37 2","pages":""},"PeriodicalIF":1.9,"publicationDate":"2023-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43223707","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Deploying vaccine distribution sites for improved accessibility and equity to support pandemic response 部署疫苗分发点，提高可及性和公平性，以支持疫情应对

IF 1.9 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2023-08-02 DOI: 10.1007/s10458-023-09614-9

George Z. Li, Ann Li, Madhav Marathe, Aravind Srinivasan, Leonidas Tsepenekas, Anil Vullikanti

引用次数: 3

Balancing fairness and efficiency in traffic routing via interpolated traffic assignment 通过插值流量分配平衡流量路由的公平性和效率

IF 1.9 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2023-08-02 DOI: 10.1007/s10458-023-09616-7

Devansh Jalota, Kiril Solovey, Matthew Tsao, Stephen Zoepf, Marco Pavone

{"title":"Balancing fairness and efficiency in traffic routing via interpolated traffic assignment","authors":"Devansh Jalota, Kiril Solovey, Matthew Tsao, Stephen Zoepf, Marco Pavone","doi":"10.1007/s10458-023-09616-7","DOIUrl":"10.1007/s10458-023-09616-7","url":null,"abstract":"<div><p>System optimum (SO) routing, wherein the total travel time of all users is minimized, is a holy grail for transportation authorities. However, SO routing may discriminate against users who incur much larger travel times than others to achieve high system efficiency, i.e., low total travel times. To address the inherent unfairness of SO routing, we study the <span>({beta })</span>-fair SO problem whose goal is to minimize the total travel time while guaranteeing a <span>({beta ge 1})</span> level of unfairness, which specifies the maximum possible ratio between the travel times of different users with shared origins and destinations. To obtain feasible solutions to the <span>({beta })</span>-fair SO problem while achieving high system efficiency, we develop a new convex program, the interpolated traffic assignment problem (I-TAP), which interpolates between a fairness-promoting and an efficiency-promoting traffic-assignment objective. We evaluate the efficacy of I-TAP through theoretical bounds on the total system travel time and level of unfairness in terms of its interpolation parameter, as well as present a numerical comparison between I-TAP and a state-of-the-art algorithm on a range of transportation networks. The numerical results indicate that our approach is faster by several orders of magnitude as compared to the benchmark algorithm, while achieving higher system efficiency for all desirable levels of unfairness. We further leverage the structure of I-TAP to develop two pricing mechanisms to collectively enforce the I-TAP solution in the presence of selfish homogeneous and heterogeneous users, respectively, that independently choose routes to minimize their own travel costs. We mention that this is the first study of pricing in the context of fair routing for general road networks (as opposed to, e.g., parallel road networks).</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"37 2","pages":""},"PeriodicalIF":1.9,"publicationDate":"2023-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47101947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 11