{"title":"在基于强化学习的多代理协作检测中学习优化状态估计","authors":"Tianlong Zhou;Tianyi Shi;Hongye Gao;Weixiong Rao","doi":"10.1109/TMC.2024.3445583","DOIUrl":null,"url":null,"abstract":"In this paper, we study the collaborative detection problem in a multi-agent environment. By exploiting onboard range-bearing sensors, mobile agents make sequential control decisions such as moving directions to gather information of movable targets. To estimate target states, i.e., target location and velocity, the classic works such as Kalman Filter (KF) and Extended Kalman Filter (EKF) impractically assume that the underlying state space model is fully known, and some recent learning-based works, i.e., KalmanNet, estimate target states alone but without estimation uncertainty, and cannot make robust control decision. To tackle such issues, we first propose a neural network-based state estimator, namely T\n<underline>W</u>\no-phase K\n<underline>AL</u>\nma\n<underline>n</u>\n Filter with \n<underline>U</u>\nncertainty quan\n<underline>T</u>\nification (WALNUT), to explicitly give both target states and estimation uncertainty. The developed multi-agent reinforcement learning (MARL) model then takes the learned target states and uncertainty as input and makes robust actions to track movable targets. Our extensive experiments demonstrate that our work outperforms the state-of-the-art by higher tracking ability and lower localization error.","PeriodicalId":50389,"journal":{"name":"IEEE Transactions on Mobile Computing","volume":"23 12","pages":"14330-14343"},"PeriodicalIF":7.7000,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Learning to Optimize State Estimation in Multi-Agent Reinforcement Learning-Based Collaborative Detection\",\"authors\":\"Tianlong Zhou;Tianyi Shi;Hongye Gao;Weixiong Rao\",\"doi\":\"10.1109/TMC.2024.3445583\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In this paper, we study the collaborative detection problem in a multi-agent environment. By exploiting onboard range-bearing sensors, mobile agents make sequential control decisions such as moving directions to gather information of movable targets. To estimate target states, i.e., target location and velocity, the classic works such as Kalman Filter (KF) and Extended Kalman Filter (EKF) impractically assume that the underlying state space model is fully known, and some recent learning-based works, i.e., KalmanNet, estimate target states alone but without estimation uncertainty, and cannot make robust control decision. To tackle such issues, we first propose a neural network-based state estimator, namely T\\n<underline>W</u>\\no-phase K\\n<underline>AL</u>\\nma\\n<underline>n</u>\\n Filter with \\n<underline>U</u>\\nncertainty quan\\n<underline>T</u>\\nification (WALNUT), to explicitly give both target states and estimation uncertainty. The developed multi-agent reinforcement learning (MARL) model then takes the learned target states and uncertainty as input and makes robust actions to track movable targets. Our extensive experiments demonstrate that our work outperforms the state-of-the-art by higher tracking ability and lower localization error.\",\"PeriodicalId\":50389,\"journal\":{\"name\":\"IEEE Transactions on Mobile Computing\",\"volume\":\"23 12\",\"pages\":\"14330-14343\"},\"PeriodicalIF\":7.7000,\"publicationDate\":\"2024-08-19\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Mobile Computing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10638830/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Mobile Computing","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10638830/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
Learning to Optimize State Estimation in Multi-Agent Reinforcement Learning-Based Collaborative Detection
In this paper, we study the collaborative detection problem in a multi-agent environment. By exploiting onboard range-bearing sensors, mobile agents make sequential control decisions such as moving directions to gather information of movable targets. To estimate target states, i.e., target location and velocity, the classic works such as Kalman Filter (KF) and Extended Kalman Filter (EKF) impractically assume that the underlying state space model is fully known, and some recent learning-based works, i.e., KalmanNet, estimate target states alone but without estimation uncertainty, and cannot make robust control decision. To tackle such issues, we first propose a neural network-based state estimator, namely T
W
o-phase K
AL
ma
n
Filter with
U
ncertainty quan
T
ification (WALNUT), to explicitly give both target states and estimation uncertainty. The developed multi-agent reinforcement learning (MARL) model then takes the learned target states and uncertainty as input and makes robust actions to track movable targets. Our extensive experiments demonstrate that our work outperforms the state-of-the-art by higher tracking ability and lower localization error.
期刊介绍:
IEEE Transactions on Mobile Computing addresses key technical issues related to various aspects of mobile computing. This includes (a) architectures, (b) support services, (c) algorithm/protocol design and analysis, (d) mobile environments, (e) mobile communication systems, (f) applications, and (g) emerging technologies. Topics of interest span a wide range, covering aspects like mobile networks and hosts, mobility management, multimedia, operating system support, power management, online and mobile environments, security, scalability, reliability, and emerging technologies such as wearable computers, body area networks, and wireless sensor networks. The journal serves as a comprehensive platform for advancements in mobile computing research.