Exploiting long-term markovian feature importance via dual attention for partially-connected differential architecture search

IF 8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS
Honggui Han , Qiyu Zhang , Fangyu Li , Yongping Du
{"title":"Exploiting long-term markovian feature importance via dual attention for partially-connected differential architecture search","authors":"Honggui Han ,&nbsp;Qiyu Zhang ,&nbsp;Fangyu Li ,&nbsp;Yongping Du","doi":"10.1016/j.engappai.2025.111476","DOIUrl":null,"url":null,"abstract":"<div><div>—Differentiable architecture search (DARTS) is implemented as a gradient-based search method for neural architecture generation. However, DARTS suffers from unbalanced competition between unweighted and weighted operations in the search phase of the supernetwork, resulting in a collapse of the search architecture. In this paper, exploiting long-term markovian feature importance via dual attention for partially-connected differential architecture search (MA-DARTS) is proposed, to overcome the excessive accumulation of unweighted operation dominance by reducing redundant features in the supernetwork. First, spatial location attention factors for different semantic groups are learned through spatial attention. The grouped attention approach contributes to capture changes in the spatial semantic importance of search features. Secondly, the channel feature importance is obtained by learning channel attention weights without dimensionality reduction through a one-dimensional convolution factor. Finally, a Markov chain-based long-term importance feature channel selection strategy is designed. This strategy dynamically transmits key features to the search space, which improves the utilization of effective feature information in search. The experimental results demonstrate that MA-DARTS effectively suppresses the problem of excessive proportion of unweighted operations during the search process, achieving better network performance while ensuring the stability of the architecture search. Meanwhile, the proposed method achieves 0.43 %, 0.68 % and 2.2 % accuracy improvement compared to DARTS on Canadian institute for advanced research CIFAR-10, CIFAR-100 and ImageNet datasets.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"158 ","pages":"Article 111476"},"PeriodicalIF":8.0000,"publicationDate":"2025-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Engineering Applications of Artificial Intelligence","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0952197625014782","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

—Differentiable architecture search (DARTS) is implemented as a gradient-based search method for neural architecture generation. However, DARTS suffers from unbalanced competition between unweighted and weighted operations in the search phase of the supernetwork, resulting in a collapse of the search architecture. In this paper, exploiting long-term markovian feature importance via dual attention for partially-connected differential architecture search (MA-DARTS) is proposed, to overcome the excessive accumulation of unweighted operation dominance by reducing redundant features in the supernetwork. First, spatial location attention factors for different semantic groups are learned through spatial attention. The grouped attention approach contributes to capture changes in the spatial semantic importance of search features. Secondly, the channel feature importance is obtained by learning channel attention weights without dimensionality reduction through a one-dimensional convolution factor. Finally, a Markov chain-based long-term importance feature channel selection strategy is designed. This strategy dynamically transmits key features to the search space, which improves the utilization of effective feature information in search. The experimental results demonstrate that MA-DARTS effectively suppresses the problem of excessive proportion of unweighted operations during the search process, achieving better network performance while ensuring the stability of the architecture search. Meanwhile, the proposed method achieves 0.43 %, 0.68 % and 2.2 % accuracy improvement compared to DARTS on Canadian institute for advanced research CIFAR-10, CIFAR-100 and ImageNet datasets.

Abstract Image

利用双重注意的长期马尔可夫特征重要性进行部分连通差分架构搜索
可微分结构搜索(DARTS)是一种基于梯度的神经结构生成搜索方法。然而,在超级网络的搜索阶段,非加权和加权操作之间存在不平衡竞争,导致搜索体系结构崩溃。本文提出了一种基于双关注的长期马尔可夫特征重要度的部分连接差分结构搜索(MA-DARTS)方法,通过减少超网络中的冗余特征,克服非加权操作优势度的过度积累。首先,通过空间注意学习不同语义组的空间位置注意因子。分组注意方法有助于捕捉搜索特征空间语义重要性的变化。其次,通过一维卷积因子在不降维的情况下学习信道注意权值,得到信道特征重要度;最后,设计了基于马尔可夫链的长期重要特征信道选择策略。该策略将关键特征动态传递到搜索空间,提高了有效特征信息在搜索中的利用率。实验结果表明,MA-DARTS有效地抑制了搜索过程中未加权操作比例过大的问题,在保证架构搜索稳定性的同时获得了更好的网络性能。同时,在加拿大高等研究院CIFAR-10、CIFAR-100和ImageNet数据集上,与DARTS相比,该方法的准确率分别提高了0.43%、0.68%和2.2%。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Engineering Applications of Artificial Intelligence
Engineering Applications of Artificial Intelligence 工程技术-工程:电子与电气
CiteScore
9.60
自引率
10.00%
发文量
505
审稿时长
68 days
期刊介绍: Artificial Intelligence (AI) is pivotal in driving the fourth industrial revolution, witnessing remarkable advancements across various machine learning methodologies. AI techniques have become indispensable tools for practicing engineers, enabling them to tackle previously insurmountable challenges. Engineering Applications of Artificial Intelligence serves as a global platform for the swift dissemination of research elucidating the practical application of AI methods across all engineering disciplines. Submitted papers are expected to present novel aspects of AI utilized in real-world engineering applications, validated using publicly available datasets to ensure the replicability of research outcomes. Join us in exploring the transformative potential of AI in engineering.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信