联合态势评估--机动意图决策的分层决策框架

Ruihai Chen, Hao Li, Guanwei Yan, Haojie Peng, Qian Zhang
{"title":"联合态势评估--机动意图决策的分层决策框架","authors":"Ruihai Chen, Hao Li, Guanwei Yan, Haojie Peng, Qian Zhang","doi":"10.1002/aisy.202300574","DOIUrl":null,"url":null,"abstract":"Decision‐making in unmanned combat aerial vehicles (UCAVs) presents a multifaceted challenge because of the complexity and dynamics of the flight environment, which leads to hurdles in training convergence, low decision validity, and the dimensionality catastrophe for decision‐making neural networks. A novel framework is proposed to address breaking down the complicated decision issues, which combines the strengths of graph convolutional networks in relation extraction with the ability of hierarchical reinforcement learning. To solve the problem of decision validity under high‐dimensional inputs, the joint framework is applied to the Maneuver Intent's decision, and a maneuver library‐based state space design method is suggested. The joint framework executes adaptable strategies and flight maneuvers to address the issue of training non‐convergence or task failure due to difficult‐to‐obtain reward signals across various scenarios. Then, the recurrent curriculum training and cross‐entropy rewards are designed to train decisions on different sub‐strategies. The experimental evaluation demonstrated more flexibility and adaptability in decision‐making problems under complex tasks compared to rule‐based and reinforcement learning baseline methods. The method proposed in this article provides a novel approach to resolving intricate decision problems, and which has certain theoretical significance and reference value for engineering applications.","PeriodicalId":7187,"journal":{"name":"Advanced Intelligent Systems","volume":"103 13","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Joint Situational Assessment‐Hierarchical Decision‐Making Framework for Maneuver Intent Decisions\",\"authors\":\"Ruihai Chen, Hao Li, Guanwei Yan, Haojie Peng, Qian Zhang\",\"doi\":\"10.1002/aisy.202300574\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Decision‐making in unmanned combat aerial vehicles (UCAVs) presents a multifaceted challenge because of the complexity and dynamics of the flight environment, which leads to hurdles in training convergence, low decision validity, and the dimensionality catastrophe for decision‐making neural networks. A novel framework is proposed to address breaking down the complicated decision issues, which combines the strengths of graph convolutional networks in relation extraction with the ability of hierarchical reinforcement learning. To solve the problem of decision validity under high‐dimensional inputs, the joint framework is applied to the Maneuver Intent's decision, and a maneuver library‐based state space design method is suggested. The joint framework executes adaptable strategies and flight maneuvers to address the issue of training non‐convergence or task failure due to difficult‐to‐obtain reward signals across various scenarios. Then, the recurrent curriculum training and cross‐entropy rewards are designed to train decisions on different sub‐strategies. The experimental evaluation demonstrated more flexibility and adaptability in decision‐making problems under complex tasks compared to rule‐based and reinforcement learning baseline methods. The method proposed in this article provides a novel approach to resolving intricate decision problems, and which has certain theoretical significance and reference value for engineering applications.\",\"PeriodicalId\":7187,\"journal\":{\"name\":\"Advanced Intelligent Systems\",\"volume\":\"103 13\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-04-21\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Advanced Intelligent Systems\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1002/aisy.202300574\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Advanced Intelligent Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1002/aisy.202300574","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

由于飞行环境的复杂性和动态性,无人战斗飞行器(UCAV)的决策面临着多方面的挑战,这导致了训练收敛性障碍、决策有效性低以及决策神经网络的维度灾难。为了解决复杂的决策问题,我们提出了一个新颖的框架,它结合了图卷积网络在关系提取方面的优势和分层强化学习的能力。为了解决高维输入下的决策有效性问题,联合框架被应用于操纵意图的决策,并提出了一种基于操纵库的状态空间设计方法。联合框架执行适应性策略和飞行操纵,以解决在不同场景下由于难以获得奖励信号而导致训练不收敛或任务失败的问题。然后,设计了循环课程训练和交叉熵奖励,以训练不同子策略的决策。实验评估表明,与基于规则和强化学习的基线方法相比,该方法在复杂任务下的决策问题中更具灵活性和适应性。本文提出的方法为解决错综复杂的决策问题提供了一种新颖的方法,具有一定的理论意义和工程应用参考价值。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Joint Situational Assessment‐Hierarchical Decision‐Making Framework for Maneuver Intent Decisions
Decision‐making in unmanned combat aerial vehicles (UCAVs) presents a multifaceted challenge because of the complexity and dynamics of the flight environment, which leads to hurdles in training convergence, low decision validity, and the dimensionality catastrophe for decision‐making neural networks. A novel framework is proposed to address breaking down the complicated decision issues, which combines the strengths of graph convolutional networks in relation extraction with the ability of hierarchical reinforcement learning. To solve the problem of decision validity under high‐dimensional inputs, the joint framework is applied to the Maneuver Intent's decision, and a maneuver library‐based state space design method is suggested. The joint framework executes adaptable strategies and flight maneuvers to address the issue of training non‐convergence or task failure due to difficult‐to‐obtain reward signals across various scenarios. Then, the recurrent curriculum training and cross‐entropy rewards are designed to train decisions on different sub‐strategies. The experimental evaluation demonstrated more flexibility and adaptability in decision‐making problems under complex tasks compared to rule‐based and reinforcement learning baseline methods. The method proposed in this article provides a novel approach to resolving intricate decision problems, and which has certain theoretical significance and reference value for engineering applications.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信