Distributed Gradient Method for Neural Network-Based Constrained $k$-Winners-Take-All

IF 6.7 2区计算机科学 Q1 ENGINEERING, MULTIDISCIPLINARY

IEEE Transactions on Network Science and Engineering Pub Date : 2024-08-15 DOI:10.1109/TNSE.2024.3443864

Xiasheng Shi;Yanxu Su;Chaoxu Mu;Changyin Sun

{"title":"Distributed Gradient Method for Neural Network-Based Constrained $k$-Winners-Take-All","authors":"Xiasheng Shi;Yanxu Su;Chaoxu Mu;Changyin Sun","doi":"10.1109/TNSE.2024.3443864","DOIUrl":null,"url":null,"abstract":"Thispaper studies the neural network-based distributed constrained \n<inline-formula><tex-math>$k$</tex-math></inline-formula>\n-winners-take-all (\n<inline-formula><tex-math>$k$</tex-math></inline-formula>\nWTA) problem, which aims to select \n<inline-formula><tex-math>$k$</tex-math></inline-formula>\n largest inputs from amount of inputs under two types of global coupled constraints. Namely, equality and inequality constrained \n<inline-formula><tex-math>$k$</tex-math></inline-formula>\nWTA problems. By selecting the proper parameter, the two constrained \n<inline-formula><tex-math>$k$</tex-math></inline-formula>\nWTA problems can be transformed into two continuous constrained quadratic programming problems. Subsequently, we propose a derivative feedback-based modified primal-dual fully distributed algorithm for the \n<inline-formula><tex-math>$k$</tex-math></inline-formula>\nWTA problem with a global coupled equality constraint by utilizing Karush-Kuhn-Tucker (KKT) conditions and the gradient flow method. In addition, the developed derivative feedback-based distributed neurodynamic method is initialization-free. Furthermore, the above method is revised via a maximal projection operator for the \n<inline-formula><tex-math>$k$</tex-math></inline-formula>\nWTA problem with a global coupled inequality constraint. The two methods are rigorously proved to asymptotically solve the distributed constrained \n<inline-formula><tex-math>$k$</tex-math></inline-formula>\nWTA models in accordance with LaSalle's invariance principle. The performance of our designed methods is tested via four simulation examples.","PeriodicalId":54229,"journal":{"name":"IEEE Transactions on Network Science and Engineering","volume":"11 6","pages":"5760-5772"},"PeriodicalIF":6.7000,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Network Science and Engineering","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10637718/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

Thispaper studies the neural network-based distributed constrained

$k$

-winners-take-all (

$k$

WTA) problem, which aims to select

$k$

largest inputs from amount of inputs under two types of global coupled constraints. Namely, equality and inequality constrained

$k$

WTA problems. By selecting the proper parameter, the two constrained

$k$

WTA problems can be transformed into two continuous constrained quadratic programming problems. Subsequently, we propose a derivative feedback-based modified primal-dual fully distributed algorithm for the

$k$

WTA problem with a global coupled equality constraint by utilizing Karush-Kuhn-Tucker (KKT) conditions and the gradient flow method. In addition, the developed derivative feedback-based distributed neurodynamic method is initialization-free. Furthermore, the above method is revised via a maximal projection operator for the

$k$

WTA problem with a global coupled inequality constraint. The two methods are rigorously proved to asymptotically solve the distributed constrained

$k$

WTA models in accordance with LaSalle's invariance principle. The performance of our designed methods is tested via four simulation examples.

查看原文本刊更多论文

基于神经网络的分布式梯度法（Distributed Gradient Method for Neural Network-based Constrained $k$-win-win-take-all

本文研究了基于神经网络的分布式受限 $k$ 赢家通吃（$k$WTA）问题，该问题旨在从两类全局耦合约束条件下的投入量中选择 $k$ 最大投入。即平等和不平等约束的 $k$WTA 问题。通过选择适当的参数，这两个受约束的 $k$WTA 问题可以转化为两个连续受约束的二次编程问题。随后，我们利用 Karush-Kuhn-Tucker（KKT）条件和梯度流方法，针对具有全局耦合相等约束的 $k$WTA 问题提出了一种基于导数反馈的修正原始双全分布式算法。此外，所开发的基于导数反馈的分布式神经动力学方法无需初始化。此外，针对具有全局耦合不等式约束的 $k$WTA 问题，通过最大投影算子对上述方法进行了修正。根据拉萨尔不变性原理，这两种方法都能渐近地求解分布式约束 $k$WTA 模型。我们通过四个模拟实例检验了所设计方法的性能。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Network Science and Engineering Engineering-Control and Systems Engineering

CiteScore

12.60

自引率

9.10%

发文量

393

期刊介绍： The proposed journal, called the IEEE Transactions on Network Science and Engineering (TNSE), is committed to timely publishing of peer-reviewed technical articles that deal with the theory and applications of network science and the interconnections among the elements in a system that form a network. In particular, the IEEE Transactions on Network Science and Engineering publishes articles on understanding, prediction, and control of structures and behaviors of networks at the fundamental level. The types of networks covered include physical or engineered networks, information networks, biological networks, semantic networks, economic networks, social networks, and ecological networks. Aimed at discovering common principles that govern network structures, network functionalities and behaviors of networks, the journal seeks articles on understanding, prediction, and control of structures and behaviors of networks. Another trans-disciplinary focus of the IEEE Transactions on Network Science and Engineering is the interactions between and co-evolution of different genres of networks.