TTSDA-YOLO:用于恶劣天气下物体检测的双训练阶段领域适应框架

IF 5.6 2区 工程技术 Q1 ENGINEERING, ELECTRICAL & ELECTRONIC
Mengmeng Zhang;Qiyu Rong;Hongyuan Jing
{"title":"TTSDA-YOLO:用于恶劣天气下物体检测的双训练阶段领域适应框架","authors":"Mengmeng Zhang;Qiyu Rong;Hongyuan Jing","doi":"10.1109/TIM.2024.3497132","DOIUrl":null,"url":null,"abstract":"Object detection plays a crucial role in the fields of autonomous driving, security surveillance, unmanned aerial vehicle (UAV), and so on. However, the performance of detectors can be drastically degraded by adverse weather conditions, such as fog, rain, and snow. This is because detectors are usually trained on images taken in clear weather conditions but tested under adverse weather conditions. There is a domain shift problem between images captured in adverse weather and those taken in clear weather. In this article, we propose a robust detection framework called two training stage domain adaptation you only look once (TTSDA-YOLO), which performs well in both normal and adverse weather conditions based on YOLOv7. We design a new training strategy that fully utilizes auxiliary domains to transfer knowledge from the source domain to the target domain. This training strategy consists of two stages. In the first training stage, we address the disparity in feature distributions between normal weather images and adverse weather images. We use a multiscale image-level domain adaptation (IDA) module to gradually adapt the normal weather domain to the adverse weather domain. In the second training stage, we make full use of the auxiliary domain by inputting it into the network as a training set. To prevent new domain shifts from being generated during the training process, we design a backbone regularization module (BRM). Extensive experimental results of the proposed TTSDA-YOLO on benchmark datasets show that our approach can significantly improve the detection performance of the network in adverse weather conditions.","PeriodicalId":13341,"journal":{"name":"IEEE Transactions on Instrumentation and Measurement","volume":"74 ","pages":"1-13"},"PeriodicalIF":5.6000,"publicationDate":"2024-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"TTSDA-YOLO: A Two Training Stage Domain Adaptation Framework for Object Detection in Adverse Weather\",\"authors\":\"Mengmeng Zhang;Qiyu Rong;Hongyuan Jing\",\"doi\":\"10.1109/TIM.2024.3497132\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Object detection plays a crucial role in the fields of autonomous driving, security surveillance, unmanned aerial vehicle (UAV), and so on. However, the performance of detectors can be drastically degraded by adverse weather conditions, such as fog, rain, and snow. This is because detectors are usually trained on images taken in clear weather conditions but tested under adverse weather conditions. There is a domain shift problem between images captured in adverse weather and those taken in clear weather. In this article, we propose a robust detection framework called two training stage domain adaptation you only look once (TTSDA-YOLO), which performs well in both normal and adverse weather conditions based on YOLOv7. We design a new training strategy that fully utilizes auxiliary domains to transfer knowledge from the source domain to the target domain. This training strategy consists of two stages. In the first training stage, we address the disparity in feature distributions between normal weather images and adverse weather images. We use a multiscale image-level domain adaptation (IDA) module to gradually adapt the normal weather domain to the adverse weather domain. In the second training stage, we make full use of the auxiliary domain by inputting it into the network as a training set. To prevent new domain shifts from being generated during the training process, we design a backbone regularization module (BRM). Extensive experimental results of the proposed TTSDA-YOLO on benchmark datasets show that our approach can significantly improve the detection performance of the network in adverse weather conditions.\",\"PeriodicalId\":13341,\"journal\":{\"name\":\"IEEE Transactions on Instrumentation and Measurement\",\"volume\":\"74 \",\"pages\":\"1-13\"},\"PeriodicalIF\":5.6000,\"publicationDate\":\"2024-11-13\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Instrumentation and Measurement\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10752537/\",\"RegionNum\":2,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Instrumentation and Measurement","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10752537/","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

摘要

物体检测在自动驾驶、安全监控、无人驾驶飞行器(UAV)等领域发挥着至关重要的作用。然而,在雾、雨、雪等恶劣天气条件下,探测器的性能会大幅下降。这是因为探测器通常是在晴朗天气条件下拍摄的图像上进行训练,但在恶劣天气条件下进行测试。在恶劣天气下拍摄的图像与晴朗天气下拍摄的图像之间存在域偏移问题。在本文中,我们以 YOLOv7 为基础,提出了一种稳健的检测框架,称为 "只看一次的两阶段训练域适应(TTSDA-YOLO)",该框架在正常和恶劣天气条件下均表现良好。我们设计了一种新的训练策略,充分利用辅助域将知识从源域转移到目标域。这种训练策略包括两个阶段。在第一个训练阶段,我们解决了正常天气图像和恶劣天气图像在特征分布上的差异。我们使用多尺度图像级域适应(IDA)模块,逐步将正常天气域适应到不利天气域。在第二个训练阶段,我们充分利用辅助域,将其作为训练集输入网络。为了防止在训练过程中产生新的域偏移,我们设计了一个骨干正则化模块(BRM)。所提出的 TTSDA-YOLO 在基准数据集上的大量实验结果表明,我们的方法可以显著提高网络在恶劣天气条件下的检测性能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
TTSDA-YOLO: A Two Training Stage Domain Adaptation Framework for Object Detection in Adverse Weather
Object detection plays a crucial role in the fields of autonomous driving, security surveillance, unmanned aerial vehicle (UAV), and so on. However, the performance of detectors can be drastically degraded by adverse weather conditions, such as fog, rain, and snow. This is because detectors are usually trained on images taken in clear weather conditions but tested under adverse weather conditions. There is a domain shift problem between images captured in adverse weather and those taken in clear weather. In this article, we propose a robust detection framework called two training stage domain adaptation you only look once (TTSDA-YOLO), which performs well in both normal and adverse weather conditions based on YOLOv7. We design a new training strategy that fully utilizes auxiliary domains to transfer knowledge from the source domain to the target domain. This training strategy consists of two stages. In the first training stage, we address the disparity in feature distributions between normal weather images and adverse weather images. We use a multiscale image-level domain adaptation (IDA) module to gradually adapt the normal weather domain to the adverse weather domain. In the second training stage, we make full use of the auxiliary domain by inputting it into the network as a training set. To prevent new domain shifts from being generated during the training process, we design a backbone regularization module (BRM). Extensive experimental results of the proposed TTSDA-YOLO on benchmark datasets show that our approach can significantly improve the detection performance of the network in adverse weather conditions.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE Transactions on Instrumentation and Measurement
IEEE Transactions on Instrumentation and Measurement 工程技术-工程:电子与电气
CiteScore
9.00
自引率
23.20%
发文量
1294
审稿时长
3.9 months
期刊介绍: Papers are sought that address innovative solutions to the development and use of electrical and electronic instruments and equipment to measure, monitor and/or record physical phenomena for the purpose of advancing measurement science, methods, functionality and applications. The scope of these papers may encompass: (1) theory, methodology, and practice of measurement; (2) design, development and evaluation of instrumentation and measurement systems and components used in generating, acquiring, conditioning and processing signals; (3) analysis, representation, display, and preservation of the information obtained from a set of measurements; and (4) scientific and technical support to establishment and maintenance of technical standards in the field of Instrumentation and Measurement.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信