LMSFF: Lightweight multi-scale feature fusion network for image recognition under resource-constrained environments

IF 7.5 1区 计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE
Yuchen Liu , Hu Liang , Shengrong Zhao
{"title":"LMSFF: Lightweight multi-scale feature fusion network for image recognition under resource-constrained environments","authors":"Yuchen Liu ,&nbsp;Hu Liang ,&nbsp;Shengrong Zhao","doi":"10.1016/j.eswa.2024.125584","DOIUrl":null,"url":null,"abstract":"<div><div>In many resource-constrained environments, recognition tasks often require efficient and fast execution. Currently, many methods designed for this field adopt a combination of convolutional operations and Vision Transformers (ViTs) to achieve comprehensive feature representation while maintaining efficient performance. However, these methods still have higher parameter counts or floating point operations (FLOPs), making it difficult to adapt more resource-constrained environments. Therefore, a lightweight Multi-Scale Feature Fusion Network (LMSFF) is proposed to address this issue. The proposed method mainly consists of three modules: lightweight local processing (LLP) modules, local–global fusion modules (LGFM), and lightweight information fusion (LIF) modules. The LLP modules, considering the issue of computational redundancy, propose a branch structure that effectively reduces parameter consumption while maintaining high performance. To capture more comprehensive contextual information, the LGFM fuses local and global features, thus enhancing the comprehensive representation of image features. The LIF extracts crucial features through pooling operations at different scales while preserving lightweight characteristics. Additionally, to enhance the model’s generalization, a new weighted loss function is introduced, which alleviates the long-tail distribution issue in real-world scenarios and improves recognition performance for rare categories. Experimental results demonstrate that LMSFF achieves better balance between recognition accuracy and resource consumption compared with other state-of-the-art lightweight hybrid models.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":null,"pages":null},"PeriodicalIF":7.5000,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Expert Systems with Applications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0957417424024515","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

Abstract

In many resource-constrained environments, recognition tasks often require efficient and fast execution. Currently, many methods designed for this field adopt a combination of convolutional operations and Vision Transformers (ViTs) to achieve comprehensive feature representation while maintaining efficient performance. However, these methods still have higher parameter counts or floating point operations (FLOPs), making it difficult to adapt more resource-constrained environments. Therefore, a lightweight Multi-Scale Feature Fusion Network (LMSFF) is proposed to address this issue. The proposed method mainly consists of three modules: lightweight local processing (LLP) modules, local–global fusion modules (LGFM), and lightweight information fusion (LIF) modules. The LLP modules, considering the issue of computational redundancy, propose a branch structure that effectively reduces parameter consumption while maintaining high performance. To capture more comprehensive contextual information, the LGFM fuses local and global features, thus enhancing the comprehensive representation of image features. The LIF extracts crucial features through pooling operations at different scales while preserving lightweight characteristics. Additionally, to enhance the model’s generalization, a new weighted loss function is introduced, which alleviates the long-tail distribution issue in real-world scenarios and improves recognition performance for rare categories. Experimental results demonstrate that LMSFF achieves better balance between recognition accuracy and resource consumption compared with other state-of-the-art lightweight hybrid models.
LMSFF:用于资源受限环境下图像识别的轻量级多尺度特征融合网络
在许多资源有限的环境中,识别任务往往需要高效快速地执行。目前,为这一领域设计的许多方法都采用卷积运算和视觉变换器(ViT)相结合的方式,在保持高效性能的同时实现全面的特征表示。然而,这些方法的参数计数或浮点运算(FLOP)仍然较高,难以适应资源有限的环境。因此,我们提出了一种轻量级多尺度特征融合网络(LMSFF)来解决这一问题。该方法主要由三个模块组成:轻量级本地处理(LLP)模块、本地-全局融合(LGFM)模块和轻量级信息融合(LIF)模块。LLP 模块考虑到计算冗余问题,提出了一种分支结构,在保持高性能的同时有效减少了参数消耗。为了捕捉更全面的上下文信息,LGFM 模块融合了局部特征和全局特征,从而增强了图像特征的全面呈现。LIF 通过不同尺度的池化操作提取关键特征,同时保留轻量级特征。此外,为了增强模型的泛化能力,还引入了新的加权损失函数,从而缓解了实际场景中的长尾分布问题,提高了稀有类别的识别性能。实验结果表明,与其他最先进的轻量级混合模型相比,LMSFF 在识别准确率和资源消耗之间实现了更好的平衡。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Expert Systems with Applications
Expert Systems with Applications 工程技术-工程:电子与电气
CiteScore
13.80
自引率
10.60%
发文量
2045
审稿时长
8.7 months
期刊介绍: Expert Systems With Applications is an international journal dedicated to the exchange of information on expert and intelligent systems used globally in industry, government, and universities. The journal emphasizes original papers covering the design, development, testing, implementation, and management of these systems, offering practical guidelines. It spans various sectors such as finance, engineering, marketing, law, project management, information management, medicine, and more. The journal also welcomes papers on multi-agent systems, knowledge management, neural networks, knowledge discovery, data mining, and other related areas, excluding applications to military/defense systems.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信