Honggui Han , Qiyu Zhang , Fangyu Li , Yongping Du
{"title":"Exploiting long-term markovian feature importance via dual attention for partially-connected differential architecture search","authors":"Honggui Han , Qiyu Zhang , Fangyu Li , Yongping Du","doi":"10.1016/j.engappai.2025.111476","DOIUrl":null,"url":null,"abstract":"<div><div>—Differentiable architecture search (DARTS) is implemented as a gradient-based search method for neural architecture generation. However, DARTS suffers from unbalanced competition between unweighted and weighted operations in the search phase of the supernetwork, resulting in a collapse of the search architecture. In this paper, exploiting long-term markovian feature importance via dual attention for partially-connected differential architecture search (MA-DARTS) is proposed, to overcome the excessive accumulation of unweighted operation dominance by reducing redundant features in the supernetwork. First, spatial location attention factors for different semantic groups are learned through spatial attention. The grouped attention approach contributes to capture changes in the spatial semantic importance of search features. Secondly, the channel feature importance is obtained by learning channel attention weights without dimensionality reduction through a one-dimensional convolution factor. Finally, a Markov chain-based long-term importance feature channel selection strategy is designed. This strategy dynamically transmits key features to the search space, which improves the utilization of effective feature information in search. The experimental results demonstrate that MA-DARTS effectively suppresses the problem of excessive proportion of unweighted operations during the search process, achieving better network performance while ensuring the stability of the architecture search. Meanwhile, the proposed method achieves 0.43 %, 0.68 % and 2.2 % accuracy improvement compared to DARTS on Canadian institute for advanced research CIFAR-10, CIFAR-100 and ImageNet datasets.</div></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":"158 ","pages":"Article 111476"},"PeriodicalIF":8.0000,"publicationDate":"2025-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Engineering Applications of Artificial Intelligence","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0952197625014782","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
—Differentiable architecture search (DARTS) is implemented as a gradient-based search method for neural architecture generation. However, DARTS suffers from unbalanced competition between unweighted and weighted operations in the search phase of the supernetwork, resulting in a collapse of the search architecture. In this paper, exploiting long-term markovian feature importance via dual attention for partially-connected differential architecture search (MA-DARTS) is proposed, to overcome the excessive accumulation of unweighted operation dominance by reducing redundant features in the supernetwork. First, spatial location attention factors for different semantic groups are learned through spatial attention. The grouped attention approach contributes to capture changes in the spatial semantic importance of search features. Secondly, the channel feature importance is obtained by learning channel attention weights without dimensionality reduction through a one-dimensional convolution factor. Finally, a Markov chain-based long-term importance feature channel selection strategy is designed. This strategy dynamically transmits key features to the search space, which improves the utilization of effective feature information in search. The experimental results demonstrate that MA-DARTS effectively suppresses the problem of excessive proportion of unweighted operations during the search process, achieving better network performance while ensuring the stability of the architecture search. Meanwhile, the proposed method achieves 0.43 %, 0.68 % and 2.2 % accuracy improvement compared to DARTS on Canadian institute for advanced research CIFAR-10, CIFAR-100 and ImageNet datasets.
期刊介绍:
Artificial Intelligence (AI) is pivotal in driving the fourth industrial revolution, witnessing remarkable advancements across various machine learning methodologies. AI techniques have become indispensable tools for practicing engineers, enabling them to tackle previously insurmountable challenges. Engineering Applications of Artificial Intelligence serves as a global platform for the swift dissemination of research elucidating the practical application of AI methods across all engineering disciplines. Submitted papers are expected to present novel aspects of AI utilized in real-world engineering applications, validated using publicly available datasets to ensure the replicability of research outcomes. Join us in exploring the transformative potential of AI in engineering.