Autonomous Agents and Multi-Agent Systems最新文献_第6页

Assimilating human feedback from autonomous vehicle interaction in reinforcement learning models 在强化学习模型中吸收来自自动驾驶汽车交互的人类反馈

IF 2 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2024-06-26 DOI: 10.1007/s10458-024-09659-4

Richard Fox, Elliot A. Ludvig

{"title":"Assimilating human feedback from autonomous vehicle interaction in reinforcement learning models","authors":"Richard Fox, Elliot A. Ludvig","doi":"10.1007/s10458-024-09659-4","DOIUrl":"10.1007/s10458-024-09659-4","url":null,"abstract":"<div><p>A significant challenge for real-world automated vehicles (AVs) is their interaction with human pedestrians. This paper develops a methodology to directly elicit the AV behaviour pedestrians find suitable by collecting quantitative data that can be used to measure and improve an algorithm's performance. Starting with a Deep Q Network (DQN) trained on a simple Pygame/Python-based pedestrian crossing environment, the reward structure was adapted to allow adjustment by human feedback. Feedback was collected by eliciting behavioural judgements collected from people in a controlled environment. The reward was shaped by the inter-action vector, decomposed into feature aspects for relevant behaviours, thereby facilitating both implicit preference selection and explicit task discovery in tandem. Using computational RL and behavioural-science techniques, we harness a formal iterative feedback loop where the rewards were repeatedly adapted based on human behavioural judgments. Experiments were conducted with 124 participants that showed strong initial improvement in the judgement of AV behaviours with the adaptive reward structure. The results indicate that the primary avenue for enhancing vehicle behaviour lies in the predictability of its movements when introduced. More broadly, recognising AV behaviours that receive favourable human judgments can pave the way for enhanced performance.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"38 2","pages":""},"PeriodicalIF":2.0,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10458-024-09659-4.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141506320","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Correction: Warmth and competence in human-agent cooperation 更正：人机合作中的温情与能力

IF 2 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2024-06-07 DOI: 10.1007/s10458-024-09654-9

Kevin R. McKee, Xuechunzi Bai, Susan T. Fiske

引用次数: 0

A framework for trust-related knowledge transfer in human–robot interaction 人机交互中与信任相关的知识转移框架

IF 2 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2024-05-29 DOI: 10.1007/s10458-024-09653-w

Mohammed Diab, Yiannis Demiris

{"title":"A framework for trust-related knowledge transfer in human–robot interaction","authors":"Mohammed Diab, Yiannis Demiris","doi":"10.1007/s10458-024-09653-w","DOIUrl":"10.1007/s10458-024-09653-w","url":null,"abstract":"<div><p>Trustworthy human–robot interaction (HRI) during activities of daily living (ADL) presents an interesting and challenging domain for assistive robots, particularly since methods for estimating the trust level of a human participant towards the assistive robot are still in their infancy. Trust is a multifaced concept which is affected by the interactions between the robot and the human, and depends, among other factors, on the history of the robot’s functionality, the task and the environmental state. In this paper, we are concerned with the challenge of trust transfer, i.e. whether experiences from interactions on a previous collaborative task can be taken into consideration in the trust level inference for a new collaborative task. This has the potential of avoiding re-computing trust levels from scratch for every new situation. The key challenge here is to automatically evaluate the similarity between the original and the novel situation, then adapt the robot’s behaviour to the novel situation using previous experience with various objects and tasks. To achieve this, we measure the semantic similarity between concepts in knowledge graphs (KGs) and adapt the robot’s actions towards a specific user based on personalised interaction histories. These actions are grounded and then verified before execution using a geometric motion planner to generate feasible trajectories in novel situations. This framework has been experimentally tested in human–robot handover tasks in different kitchen scene contexts. We conclude that trust-related knowledge positively influences and improves collaboration in both performance and time aspects.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"38 1","pages":""},"PeriodicalIF":2.0,"publicationDate":"2024-05-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10458-024-09653-w.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141168330","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Warmth and competence in human-agent cooperation 人机合作中的温情与能力

IF 2 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2024-05-22 DOI: 10.1007/s10458-024-09649-6

Kevin R. McKee, Xuechunzi Bai, Susan T. Fiske

{"title":"Warmth and competence in human-agent cooperation","authors":"Kevin R. McKee, Xuechunzi Bai, Susan T. Fiske","doi":"10.1007/s10458-024-09649-6","DOIUrl":"10.1007/s10458-024-09649-6","url":null,"abstract":"<div><p>Interaction and cooperation with humans are overarching aspirations of artificial intelligence research. Recent studies demonstrate that AI agents trained with deep reinforcement learning are capable of collaborating with humans. These studies primarily evaluate human compatibility through “objective” metrics such as task performance, obscuring potential variation in the levels of trust and subjective preference that different agents garner. To better understand the factors shaping subjective preferences in human-agent cooperation, we train deep reinforcement learning agents in Coins, a two-player social dilemma. We recruit <span>(N = 501)</span> participants for a human-agent cooperation study and measure their impressions of the agents they encounter. Participants’ perceptions of warmth and competence predict their stated preferences for different agents, above and beyond objective performance metrics. Drawing inspiration from social science and biology research, we subsequently implement a new “partner choice” framework to elicit <i>revealed</i> preferences: after playing an episode with an agent, participants are asked whether they would like to play the next episode with the same agent or to play alone. As with stated preferences, social perception better predicts participants’ revealed preferences than does objective performance. Given these results, we recommend human-agent interaction researchers routinely incorporate the measurement of social perception and subjective preferences into their studies.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"38 1","pages":""},"PeriodicalIF":2.0,"publicationDate":"2024-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10458-024-09649-6.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141147269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Majority opinion diffusion: when tie-breaking rule matters 多数意见扩散：当打破平局规则很重要时

IF 2 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2024-05-20 DOI: 10.1007/s10458-024-09651-y

Ahad N. Zehmakan

{"title":"Majority opinion diffusion: when tie-breaking rule matters","authors":"Ahad N. Zehmakan","doi":"10.1007/s10458-024-09651-y","DOIUrl":"10.1007/s10458-024-09651-y","url":null,"abstract":"<div><p>Consider a graph <i>G</i>, which represents a social network, and assume that initially each node is either blue or white (corresponding to its opinion on a certain topic). In each round, all nodes simultaneously update their color to the most frequent color in their neighborhood. This is called the Majority Model (MM) if a node keeps its color in case of a tie and the Random Majority Model (RMM) if it chooses blue with probability 1/2 and white otherwise. We study the convergence properties of the above models, including stabilization time, periodicity, and the number of stable configurations. In particular, we prove that the stabilization time in RMM can be exponential in the size of the graph, which is in contrast with the previously known polynomial bound on the stabilization time of MM. We provide some bounds on the minimum size of a winning set, which is a set of nodes whose agreement on a color in the initial coloring enforces the process to end in a coloring where all nodes share that color. Furthermore, we calculate the expected final number of blue nodes for a random initial coloring, where each node is colored blue independently with some fixed probability, on cycle graphs. Finally, we conduct some experiments which complement our theoretical findings and also let us investigate other aspects of the models.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"38 1","pages":""},"PeriodicalIF":2.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10458-024-09651-y.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141121750","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Logic-based cognitive planning for conversational agents 对话式代理基于逻辑的认知规划

IF 2 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2024-05-20 DOI: 10.1007/s10458-024-09646-9

Jorge Luis Fernandez Davila, Dominique Longin, Emiliano Lorini, Frédéric Maris

引用次数: 0

Tackling school segregation with transportation network interventions: an agent-based modelling approach 利用交通网络干预措施解决学校隔离问题：基于代理的建模方法

IF 2 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2024-05-20 DOI: 10.1007/s10458-024-09652-x

Dimitris Michailidis, Mayesha Tasnim, Sennay Ghebreab, Fernando P. Santos

{"title":"Tackling school segregation with transportation network interventions: an agent-based modelling approach","authors":"Dimitris Michailidis, Mayesha Tasnim, Sennay Ghebreab, Fernando P. Santos","doi":"10.1007/s10458-024-09652-x","DOIUrl":"10.1007/s10458-024-09652-x","url":null,"abstract":"<div><p>We address the emerging challenge of school segregation within the context of free school choice systems. Households take into account both proximity and demographic composition when deciding on which schools to send their children to, potentially exacerbating residential segregation. This raises an important question: can we strategically intervene in transportation networks to enhance school access and mitigate segregation? In this paper, we propose a novel, network agent-based model to explore this question. Through simulations in both synthetic and real-world networks, we demonstrate that enhancing school accessibility via transportation network interventions can lead to a reduction in school segregation, under specific conditions. We introduce group-based network centrality measures and show that increasing the centrality of certain neighborhood nodes with respect to a transportation network can be an effective strategy for strategic interventions. We conduct experiments in two synthetic network environments, as well as in an environment based on real-world data from Amsterdam, the Netherlands. In both cases, we simulate a population of representative agents emulating real citizens’ schooling preferences, and we assume that agents belong to two different groups (e.g., based on migration background). We show that, under specific homophily regimes in the population, school segregation can be reduced by up to 35%. Our proposed framework provides the foundation to explore how citizens’ preferences, school capacity, and public transportation can shape patterns of urban segregation.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"38 1","pages":""},"PeriodicalIF":2.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10458-024-09652-x.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141120455","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Computing balanced solutions for large international kidney exchange schemes 计算大型国际换肾计划的平衡解决方案

IF 2 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2024-05-16 DOI: 10.1007/s10458-024-09645-w

Márton Benedek, Péter Biró, Daniel Paulusma, Xin Ye

{"title":"Computing balanced solutions for large international kidney exchange schemes","authors":"Márton Benedek, Péter Biró, Daniel Paulusma, Xin Ye","doi":"10.1007/s10458-024-09645-w","DOIUrl":"10.1007/s10458-024-09645-w","url":null,"abstract":"<div><p>To overcome incompatibility issues, kidney patients may swap their donors. In international kidney exchange programmes (IKEPs), countries merge their national patient–donor pools. We consider a recently introduced credit system. In each round, countries are given an initial “fair” allocation of the total number of kidney transplants. This allocation is adjusted by a credit function yielding a target allocation. The goal is to find a solution that approaches the target allocation as closely as possible, to ensure long-term stability of the international pool. As solutions, we use maximum matchings that lexicographically minimize the country deviations from the target allocation. We perform, for the first time, a computational study for a <i>large</i> number of countries. For the initial allocations we use two easy-to-compute solution concepts, the benefit value and the contribution value, and four classical but hard-to-compute concepts, the Shapley value, nucleolus, Banzhaf value and tau value. By using state-of-the-art software we show that the latter four concepts are now within reach for IKEPs of up to fifteen countries. Our experiments show that using lexicographically minimal maximum matchings instead of ones that only minimize the largest deviation from the target allocation (as previously done) may make an IKEP up to 54% more balanced.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"38 1","pages":""},"PeriodicalIF":2.0,"publicationDate":"2024-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10458-024-09645-w.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141062252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Offline policy reuse-guided anytime online collective multiagent planning and its application to mobility-on-demand systems 离线策略重用指导下的随时在线多代理集体规划及其在按需移动系统中的应用

IF 2 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2024-05-16 DOI: 10.1007/s10458-024-09650-z

Wanyuan Wang, Qian Che, Yifeng Zhou, Weiwei Wu, Bo An, Yichuan Jiang

{"title":"Offline policy reuse-guided anytime online collective multiagent planning and its application to mobility-on-demand systems","authors":"Wanyuan Wang, Qian Che, Yifeng Zhou, Weiwei Wu, Bo An, Yichuan Jiang","doi":"10.1007/s10458-024-09650-z","DOIUrl":"10.1007/s10458-024-09650-z","url":null,"abstract":"<div><p>The popularity of mobility-on-demand (MoD) systems boosts online collective multiagent planning (Online_CMP), where spatially distributed servicing agents are planned to meet dynamically arriving demands. For city-scale MoDs with a fleet of agents, Online_CMP methods must make a tradeoff between computation time (i.e., real-time) and solution quality (i.e., the number of demands served). Directly using an offline policy can guarantee real-time, but cannot be dynamically adjusted to real agent and demand distributions. Search-based online planning methods are adaptive, but are computationally expensive and cannot scale up. In this paper, we propose a principled Online_CMP method, which reuses and improves the offline policy in an anytime manner. We first model MoDs as a collective Markov Decision Process (<span>({mathbb {C}})</span>-MDP) where the collective behavior of agents affects the joint reward. Given the <span>({mathbb {C}})</span>-MDP model, we propose a novel state value function to evaluate the policy, and a gradient ascent (GA) technique to improve the policy. We further show that offline GA-based policy iteration (GA-PI) can converge to global optima of <span>({mathbb {C}})</span>-MDP under certain conditions. Finally, with real-time information, the offline policy is used as the default plan, GA-PI is used to improve it and generate an online plan. Experimental results show that our offline policy reuse-guided Online_CMP method significantly outperforms standard online multiagent planning methods on MoD systems like ride-sharing and security traffic patrolling in terms of computation time and solution quality.</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"38 1","pages":""},"PeriodicalIF":2.0,"publicationDate":"2024-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140966950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Controller synthesis for linear temporal logic and steady-state specifications 线性时序逻辑和稳态规范的控制器合成

IF 2 3区计算机科学

Autonomous Agents and Multi-Agent Systems Pub Date : 2024-05-03 DOI: 10.1007/s10458-024-09648-7

Alvaro Velasquez, Ismail Alkhouri, Andre Beckus, Ashutosh Trivedi, George Atia

{"title":"Controller synthesis for linear temporal logic and steady-state specifications","authors":"Alvaro Velasquez, Ismail Alkhouri, Andre Beckus, Ashutosh Trivedi, George Atia","doi":"10.1007/s10458-024-09648-7","DOIUrl":"10.1007/s10458-024-09648-7","url":null,"abstract":"<div><p>The problem of deriving decision-making policies, subject to some formal specification of behavior, has been well-studied in the control synthesis, reinforcement learning, and planning communities. Such problems are typically framed in the context of a non-deterministic decision process, the non-determinism of which is optimally resolved by the computed policy. In this paper, we explore the derivation of such policies in Markov decision processes (MDPs) subject to two types of formal specifications. First, we consider steady-state specifications that reason about the infinite-frequency behavior of the resulting agent. This behavior corresponds to the frequency with which an agent visits each state as it follows its decision-making policy indefinitely. Second, we examine the infinite-trace behavior of the agent by imposing Linear Temporal Logic (LTL) constraints on the behavior induced by the resulting policy. We present an algorithm to find a deterministic policy satisfying LTL and steady-state constraints by characterizing the solutions as an integer linear program (ILP) and experimentally evaluate our approach. In our experimental results section, we evaluate the proposed ILP using MDPs with stochastic and deterministic transitions.\u0000</p></div>","PeriodicalId":55586,"journal":{"name":"Autonomous Agents and Multi-Agent Systems","volume":"38 1","pages":""},"PeriodicalIF":2.0,"publicationDate":"2024-05-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140889450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0