一种CNN与视觉变压器(Vision Transformer, ViT)相结合的互补目标检测方法

Yibo Gao, NiangWang Wang
{"title":"一种CNN与视觉变压器(Vision Transformer, ViT)相结合的互补目标检测方法","authors":"Yibo Gao, NiangWang Wang","doi":"10.1109/isoirs57349.2022.00009","DOIUrl":null,"url":null,"abstract":"The object detection method based on CNN is the mainstream method in the field of object detection because of its special structure of hierarchical and gradual extraction of local features, which can simultaneously consider low-level geometric features and high-level semantic features. However, this structure does not make full use of the global information of the image, resulting in classification errors when the features are limited or fuzzy. To solve the above problems, this paper explores a complementary feature extraction backbone via integrating CNN with Vision Transformer(ViT), and designs a shallow ViT structure to interact features of the proposals in a two-stage object detector with the image background feature to realize global modeling and feature alignment. In addition, according to the particularity of the supplementary structure, a segmented training strategy is designed. This strategy ensures that the model can extract the features together, and maximize the independence of their respective structures, giving full play to the advantages brought by different feature extraction methods. The model is verified on the COCO and PASCAL VOC Datasets. Through the experimental results and feature visualization analysis, it can be concluded that the mAP values are higher than CNN-based and ViT based detectors on the premise of adding limited parameters, which proves the effectiveness of the method.","PeriodicalId":405065,"journal":{"name":"2022 International Symposium on Intelligent Robotics and Systems (ISoIRS)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A complementary object detection method via integrating CNN with Vision Transformer(ViT)\",\"authors\":\"Yibo Gao, NiangWang Wang\",\"doi\":\"10.1109/isoirs57349.2022.00009\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The object detection method based on CNN is the mainstream method in the field of object detection because of its special structure of hierarchical and gradual extraction of local features, which can simultaneously consider low-level geometric features and high-level semantic features. However, this structure does not make full use of the global information of the image, resulting in classification errors when the features are limited or fuzzy. To solve the above problems, this paper explores a complementary feature extraction backbone via integrating CNN with Vision Transformer(ViT), and designs a shallow ViT structure to interact features of the proposals in a two-stage object detector with the image background feature to realize global modeling and feature alignment. In addition, according to the particularity of the supplementary structure, a segmented training strategy is designed. This strategy ensures that the model can extract the features together, and maximize the independence of their respective structures, giving full play to the advantages brought by different feature extraction methods. The model is verified on the COCO and PASCAL VOC Datasets. Through the experimental results and feature visualization analysis, it can be concluded that the mAP values are higher than CNN-based and ViT based detectors on the premise of adding limited parameters, which proves the effectiveness of the method.\",\"PeriodicalId\":405065,\"journal\":{\"name\":\"2022 International Symposium on Intelligent Robotics and Systems (ISoIRS)\",\"volume\":\"57 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2022-10-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2022 International Symposium on Intelligent Robotics and Systems (ISoIRS)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/isoirs57349.2022.00009\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2022 International Symposium on Intelligent Robotics and Systems (ISoIRS)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/isoirs57349.2022.00009","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

基于CNN的目标检测方法由于其分层渐进提取局部特征的特殊结构,可以同时考虑低级几何特征和高级语义特征,是目标检测领域的主流方法。然而,这种结构没有充分利用图像的全局信息,当特征有限或模糊时,会导致分类错误。针对上述问题,本文通过将CNN与视觉变换(Vision Transformer, ViT)相结合,探索互补的特征提取主干,并设计了一种浅层ViT结构,将两级目标检测器中各提案的特征与图像背景特征进行交互,实现全局建模和特征对齐。此外,根据辅助结构的特殊性,设计了分段训练策略。该策略保证了模型能够同时提取特征,并最大限度地发挥其各自结构的独立性,充分发挥不同特征提取方法的优势。在COCO和PASCAL VOC数据集上对模型进行了验证。通过实验结果和特征可视化分析,可以得出在添加有限参数的前提下,mAP值高于基于cnn和基于ViT的检测器,证明了该方法的有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
A complementary object detection method via integrating CNN with Vision Transformer(ViT)
The object detection method based on CNN is the mainstream method in the field of object detection because of its special structure of hierarchical and gradual extraction of local features, which can simultaneously consider low-level geometric features and high-level semantic features. However, this structure does not make full use of the global information of the image, resulting in classification errors when the features are limited or fuzzy. To solve the above problems, this paper explores a complementary feature extraction backbone via integrating CNN with Vision Transformer(ViT), and designs a shallow ViT structure to interact features of the proposals in a two-stage object detector with the image background feature to realize global modeling and feature alignment. In addition, according to the particularity of the supplementary structure, a segmented training strategy is designed. This strategy ensures that the model can extract the features together, and maximize the independence of their respective structures, giving full play to the advantages brought by different feature extraction methods. The model is verified on the COCO and PASCAL VOC Datasets. Through the experimental results and feature visualization analysis, it can be concluded that the mAP values are higher than CNN-based and ViT based detectors on the premise of adding limited parameters, which proves the effectiveness of the method.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信