{"title":"LMSFF: Lightweight multi-scale feature fusion network for image recognition under resource-constrained environments","authors":"Yuchen Liu , Hu Liang , Shengrong Zhao","doi":"10.1016/j.eswa.2024.125584","DOIUrl":null,"url":null,"abstract":"<div><div>In many resource-constrained environments, recognition tasks often require efficient and fast execution. Currently, many methods designed for this field adopt a combination of convolutional operations and Vision Transformers (ViTs) to achieve comprehensive feature representation while maintaining efficient performance. However, these methods still have higher parameter counts or floating point operations (FLOPs), making it difficult to adapt more resource-constrained environments. Therefore, a lightweight Multi-Scale Feature Fusion Network (LMSFF) is proposed to address this issue. The proposed method mainly consists of three modules: lightweight local processing (LLP) modules, local–global fusion modules (LGFM), and lightweight information fusion (LIF) modules. The LLP modules, considering the issue of computational redundancy, propose a branch structure that effectively reduces parameter consumption while maintaining high performance. To capture more comprehensive contextual information, the LGFM fuses local and global features, thus enhancing the comprehensive representation of image features. The LIF extracts crucial features through pooling operations at different scales while preserving lightweight characteristics. Additionally, to enhance the model’s generalization, a new weighted loss function is introduced, which alleviates the long-tail distribution issue in real-world scenarios and improves recognition performance for rare categories. Experimental results demonstrate that LMSFF achieves better balance between recognition accuracy and resource consumption compared with other state-of-the-art lightweight hybrid models.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":null,"pages":null},"PeriodicalIF":7.5000,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Expert Systems with Applications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0957417424024515","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
In many resource-constrained environments, recognition tasks often require efficient and fast execution. Currently, many methods designed for this field adopt a combination of convolutional operations and Vision Transformers (ViTs) to achieve comprehensive feature representation while maintaining efficient performance. However, these methods still have higher parameter counts or floating point operations (FLOPs), making it difficult to adapt more resource-constrained environments. Therefore, a lightweight Multi-Scale Feature Fusion Network (LMSFF) is proposed to address this issue. The proposed method mainly consists of three modules: lightweight local processing (LLP) modules, local–global fusion modules (LGFM), and lightweight information fusion (LIF) modules. The LLP modules, considering the issue of computational redundancy, propose a branch structure that effectively reduces parameter consumption while maintaining high performance. To capture more comprehensive contextual information, the LGFM fuses local and global features, thus enhancing the comprehensive representation of image features. The LIF extracts crucial features through pooling operations at different scales while preserving lightweight characteristics. Additionally, to enhance the model’s generalization, a new weighted loss function is introduced, which alleviates the long-tail distribution issue in real-world scenarios and improves recognition performance for rare categories. Experimental results demonstrate that LMSFF achieves better balance between recognition accuracy and resource consumption compared with other state-of-the-art lightweight hybrid models.
期刊介绍:
Expert Systems With Applications is an international journal dedicated to the exchange of information on expert and intelligent systems used globally in industry, government, and universities. The journal emphasizes original papers covering the design, development, testing, implementation, and management of these systems, offering practical guidelines. It spans various sectors such as finance, engineering, marketing, law, project management, information management, medicine, and more. The journal also welcomes papers on multi-agent systems, knowledge management, neural networks, knowledge discovery, data mining, and other related areas, excluding applications to military/defense systems.