arXiv - CS - Multiagent Systems最新文献_第8页

Improving Global Parameter-sharing in Physically Heterogeneous Multi-agent Reinforcement Learning with Unified Action Space 利用统一行动空间改进物理异构多代理强化学习中的全局参数共享

arXiv - CS - Multiagent Systems Pub Date : 2024-08-14 DOI: arxiv-2408.07395

Xiaoyang Yu, Youfang Lin, Shuo Wang, Kai Lv, Sheng Han

{"title":"Improving Global Parameter-sharing in Physically Heterogeneous Multi-agent Reinforcement Learning with Unified Action Space","authors":"Xiaoyang Yu, Youfang Lin, Shuo Wang, Kai Lv, Sheng Han","doi":"arxiv-2408.07395","DOIUrl":"https://doi.org/arxiv-2408.07395","url":null,"abstract":"In a multi-agent system (MAS), action semantics indicates the different\u0000influences of agents' actions toward other entities, and can be used to divide\u0000agents into groups in a physically heterogeneous MAS. Previous multi-agent\u0000reinforcement learning (MARL) algorithms apply global parameter-sharing across\u0000different types of heterogeneous agents without careful discrimination of\u0000different action semantics. This common implementation decreases the\u0000cooperation and coordination between agents in complex situations. However,\u0000fully independent agent parameters dramatically increase the computational cost\u0000and training difficulty. In order to benefit from the usage of different action\u0000semantics while also maintaining a proper parameter-sharing structure, we\u0000introduce the Unified Action Space (UAS) to fulfill the requirement. The UAS is\u0000the union set of all agent actions with different semantics. All agents first\u0000calculate their unified representation in the UAS, and then generate their\u0000heterogeneous action policies using different available-action-masks. To\u0000further improve the training of extra UAS parameters, we introduce a\u0000Cross-Group Inverse (CGI) loss to predict other groups' agent policies with the\u0000trajectory information. As a universal method for solving the physically\u0000heterogeneous MARL problem, we implement the UAS adding to both value-based and\u0000policy-based MARL algorithms, and propose two practical algorithms: U-QMIX and\u0000U-MAPPO. Experimental results in the SMAC environment prove the effectiveness\u0000of both U-QMIX and U-MAPPO compared with several state-of-the-art MARL methods.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"19 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190436","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-Agent Continuous Control with Generative Flow Networks 多代理连续控制与生成流网络

arXiv - CS - Multiagent Systems Pub Date : 2024-08-13 DOI: arxiv-2408.06920

Shuang Luo, Yinchuan Li, Shunyu Liu, Xu Zhang, Yunfeng Shao, Chao Wu

{"title":"Multi-Agent Continuous Control with Generative Flow Networks","authors":"Shuang Luo, Yinchuan Li, Shunyu Liu, Xu Zhang, Yunfeng Shao, Chao Wu","doi":"arxiv-2408.06920","DOIUrl":"https://doi.org/arxiv-2408.06920","url":null,"abstract":"Generative Flow Networks (GFlowNets) aim to generate diverse trajectories\u0000from a distribution in which the final states of the trajectories are\u0000proportional to the reward, serving as a powerful alternative to reinforcement\u0000learning for exploratory control tasks. However, the individual-flow matching\u0000constraint in GFlowNets limits their applications for multi-agent systems,\u0000especially continuous joint-control problems. In this paper, we propose a novel\u0000Multi-Agent generative Continuous Flow Networks (MACFN) method to enable\u0000multiple agents to perform cooperative exploration for various compositional\u0000continuous objects. Technically, MACFN trains decentralized\u0000individual-flow-based policies in a centralized global-flow-based matching\u0000fashion. During centralized training, MACFN introduces a continuous flow\u0000decomposition network to deduce the flow contributions of each agent in the\u0000presence of only global rewards. Then agents can deliver actions solely based\u0000on their assigned local flow in a decentralized way, forming a joint policy\u0000distribution proportional to the rewards. To guarantee the expressiveness of\u0000continuous flow decomposition, we theoretically derive a consistency condition\u0000on the decomposition network. Experimental results demonstrate that the\u0000proposed method yields results superior to the state-of-the-art counterparts\u0000and better exploration capability. Our code is available at\u0000https://github.com/isluoshuang/MACFN.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"29 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142224742","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Optimizing RAG Techniques for Automotive Industry PDF Chatbots: A Case Study with Locally Deployed Ollama Models 优化汽车行业 PDF 聊天机器人的 RAG 技术：本地部署的 Ollama 模型案例研究

arXiv - CS - Multiagent Systems Pub Date : 2024-08-12 DOI: arxiv-2408.05933

Fei Liu, Zejun Kang, Xing Han

{"title":"Optimizing RAG Techniques for Automotive Industry PDF Chatbots: A Case Study with Locally Deployed Ollama Models","authors":"Fei Liu, Zejun Kang, Xing Han","doi":"arxiv-2408.05933","DOIUrl":"https://doi.org/arxiv-2408.05933","url":null,"abstract":"With the growing demand for offline PDF chatbots in automotive industrial\u0000production environments, optimizing the deployment of large language models\u0000(LLMs) in local, low-performance settings has become increasingly important.\u0000This study focuses on enhancing Retrieval-Augmented Generation (RAG) techniques\u0000for processing complex automotive industry documents using locally deployed\u0000Ollama models. Based on the Langchain framework, we propose a multi-dimensional\u0000optimization approach for Ollama's local RAG implementation. Our method\u0000addresses key challenges in automotive document processing, including\u0000multi-column layouts and technical specifications. We introduce improvements in\u0000PDF processing, retrieval mechanisms, and context compression, tailored to the\u0000unique characteristics of automotive industry documents. Additionally, we\u0000design custom classes supporting embedding pipelines and an agent supporting\u0000self-RAG based on LangGraph best practices. To evaluate our approach, we\u0000constructed a proprietary dataset comprising typical automotive industry\u0000documents, including technical reports and corporate regulations. We compared\u0000our optimized RAG model and self-RAG agent against a naive RAG baseline across\u0000three datasets: our automotive industry dataset, QReCC, and CoQA. Results\u0000demonstrate significant improvements in context precision, context recall,\u0000answer relevancy, and faithfulness, with particularly notable performance on\u0000the automotive industry dataset. Our optimization scheme provides an effective\u0000solution for deploying local RAG systems in the automotive sector, addressing\u0000the specific needs of PDF chatbots in industrial production environments. This\u0000research has important implications for advancing information processing and\u0000intelligent production in the automotive industry.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"113 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190434","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

QTypeMix: Enhancing Multi-Agent Cooperative Strategies through Heterogeneous and Homogeneous Value Decomposition QTypeMix：通过异质和同质价值分解增强多代理合作策略

arXiv - CS - Multiagent Systems Pub Date : 2024-08-12 DOI: arxiv-2408.07098

Songchen Fu, Shaojing Zhao, Ta Li, YongHong Yan

{"title":"QTypeMix: Enhancing Multi-Agent Cooperative Strategies through Heterogeneous and Homogeneous Value Decomposition","authors":"Songchen Fu, Shaojing Zhao, Ta Li, YongHong Yan","doi":"arxiv-2408.07098","DOIUrl":"https://doi.org/arxiv-2408.07098","url":null,"abstract":"In multi-agent cooperative tasks, the presence of heterogeneous agents is\u0000familiar. Compared to cooperation among homogeneous agents, collaboration\u0000requires considering the best-suited sub-tasks for each agent. However, the\u0000operation of multi-agent systems often involves a large amount of complex\u0000interaction information, making it more challenging to learn heterogeneous\u0000strategies. Related multi-agent reinforcement learning methods sometimes use\u0000grouping mechanisms to form smaller cooperative groups or leverage prior domain\u0000knowledge to learn strategies for different roles. In contrast, agents should\u0000learn deeper role features without relying on additional information.\u0000Therefore, we propose QTypeMix, which divides the value decomposition process\u0000into homogeneous and heterogeneous stages. QTypeMix learns to extract type\u0000features from local historical observations through the TE loss. In addition,\u0000we introduce advanced network structures containing attention mechanisms and\u0000hypernets to enhance the representation capability and achieve the value\u0000decomposition process. The results of testing the proposed method on 14 maps\u0000from SMAC and SMACv2 show that QTypeMix achieves state-of-the-art performance\u0000in tasks of varying difficulty.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"31 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190432","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Distributed Stackelberg Strategies in State-based Potential Games for Autonomous Decentralized Learning Manufacturing Systems 自主分散学习制造系统基于状态的潜能博弈中的分布式斯塔克尔伯格策略

arXiv - CS - Multiagent Systems Pub Date : 2024-08-12 DOI: arxiv-2408.06397

Steve Yuwono, Dorothea Schwung, Andreas Schwung

{"title":"Distributed Stackelberg Strategies in State-based Potential Games for Autonomous Decentralized Learning Manufacturing Systems","authors":"Steve Yuwono, Dorothea Schwung, Andreas Schwung","doi":"arxiv-2408.06397","DOIUrl":"https://doi.org/arxiv-2408.06397","url":null,"abstract":"This article describes a novel game structure for autonomously optimizing\u0000decentralized manufacturing systems with multi-objective optimization\u0000challenges, namely Distributed Stackelberg Strategies in State-Based Potential\u0000Games (DS2-SbPG). DS2-SbPG integrates potential games and Stackelberg games,\u0000which improves the cooperative trade-off capabilities of potential games and\u0000the multi-objective optimization handling by Stackelberg games. Notably, all\u0000training procedures remain conducted in a fully distributed manner. DS2-SbPG\u0000offers a promising solution to finding optimal trade-offs between objectives by\u0000eliminating the complexities of setting up combined objective optimization\u0000functions for individual players in self-learning domains, particularly in\u0000real-world industrial settings with diverse and numerous objectives between the\u0000sub-systems. We further prove that DS2-SbPG constitutes a dynamic potential\u0000game that results in corresponding converge guarantees. Experimental validation\u0000conducted on a laboratory-scale testbed highlights the efficacy of DS2-SbPG and\u0000its two variants, such as DS2-SbPG for single-leader-follower and Stack\u0000DS2-SbPG for multi-leader-follower. The results show significant reductions in\u0000power consumption and improvements in overall performance, which signals the\u0000potential of DS2-SbPG in real-world applications.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"8 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190433","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Hierarchical Multi-Armed Bandits for the Concurrent Intelligent Tutoring of Concepts and Problems of Varying Difficulty Levels 用于不同难度概念和问题并发智能辅导的分层多臂匪帮

arXiv - CS - Multiagent Systems Pub Date : 2024-08-10 DOI: arxiv-2408.07208

Blake Castleman, Uzay Macar, Ansaf Salleb-Aouissi

{"title":"Hierarchical Multi-Armed Bandits for the Concurrent Intelligent Tutoring of Concepts and Problems of Varying Difficulty Levels","authors":"Blake Castleman, Uzay Macar, Ansaf Salleb-Aouissi","doi":"arxiv-2408.07208","DOIUrl":"https://doi.org/arxiv-2408.07208","url":null,"abstract":"Remote education has proliferated in the twenty-first century, yielding rise\u0000to intelligent tutoring systems. In particular, research has found multi-armed\u0000bandit (MAB) intelligent tutors to have notable abilities in traversing the\u0000exploration-exploitation trade-off landscape for student problem\u0000recommendations. Prior literature, however, contains a significant lack of\u0000open-sourced MAB intelligent tutors, which impedes potential applications of\u0000these educational MAB recommendation systems. In this paper, we combine recent\u0000literature on MAB intelligent tutoring techniques into an open-sourced and\u0000simply deployable hierarchical MAB algorithm, capable of progressing students\u0000concurrently through concepts and problems, determining ideal recommended\u0000problem difficulties, and assessing latent memory decay. We evaluate our\u0000algorithm using simulated groups of 500 students, utilizing Bayesian Knowledge\u0000Tracing to estimate students' content mastery. Results suggest that our\u0000algorithm, when turned difficulty-agnostic, significantly boosts student\u0000success, and that the further addition of problem-difficulty adaptation notably\u0000improves this metric.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"43 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Performative Prediction on Games and Mechanism Design 关于游戏和机制设计的表演性预测

arXiv - CS - Multiagent Systems Pub Date : 2024-08-09 DOI: arxiv-2408.05146

António Góis, Mehrnaz Mofakhami, Fernando P. Santos, Simon Lacoste-Julien, Gauthier Gidel

引用次数: 0

Performance Prediction of Hub-Based Swarms 基于集线器的蜂群性能预测

arXiv - CS - Multiagent Systems Pub Date : 2024-08-09 DOI: arxiv-2408.04822

Puneet Jain, Chaitanya Dwivedi, Vigynesh Bhatt, Nick Smith, Michael A Goodrich

引用次数: 0

Learning Fair Cooperation in Mixed-Motive Games with Indirect Reciprocity 在具有间接互惠性的混合动机游戏中学习公平合作

arXiv - CS - Multiagent Systems Pub Date : 2024-08-08 DOI: arxiv-2408.04549

Martin Smit, Fernando P. Santos

{"title":"Learning Fair Cooperation in Mixed-Motive Games with Indirect Reciprocity","authors":"Martin Smit, Fernando P. Santos","doi":"arxiv-2408.04549","DOIUrl":"https://doi.org/arxiv-2408.04549","url":null,"abstract":"Altruistic cooperation is costly yet socially desirable. As a result, agents\u0000struggle to learn cooperative policies through independent reinforcement\u0000learning (RL). Indirect reciprocity, where agents consider their interaction\u0000partner's reputation, has been shown to stabilise cooperation in homogeneous,\u0000idealised populations. However, more realistic settings are comprised of\u0000heterogeneous agents with different characteristics and group-based social\u0000identities. We study cooperation when agents are stratified into two such\u0000groups, and allow reputation updates and actions to depend on group\u0000information. We consider two modelling approaches: evolutionary game theory,\u0000where we comprehensively search for social norms (i.e., rules to assign\u0000reputations) leading to cooperation and fairness; and RL, where we consider how\u0000the stochastic dynamics of policy learning affects the analytically identified\u0000equilibria. We observe that a defecting majority leads the minority group to\u0000defect, but not the inverse. Moreover, changing the norms that judge in and\u0000out-group interactions can steer a system towards either fair or unfair\u0000cooperation. This is made clearer when moving beyond equilibrium analysis to\u0000independent RL agents, where convergence to fair cooperation occurs with a\u0000narrower set of norms. Our results highlight that, in heterogeneous populations\u0000with reputations, carefully defining interaction norms is fundamental to tackle\u0000both dilemmas of cooperation and of fairness.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"13 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141949331","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Emergence in Multi-Agent Systems: A Safety Perspective 多代理系统中的涌现：安全视角

arXiv - CS - Multiagent Systems Pub Date : 2024-08-08 DOI: arxiv-2408.04514

Philipp Altmann, Julian Schönberger, Steffen Illium, Maximilian Zorn, Fabian Ritz, Tom Haider, Simon Burton, Thomas Gabor

引用次数: 0