{"title":"TTSDA-YOLO:用于恶劣天气下物体检测的双训练阶段领域适应框架","authors":"Mengmeng Zhang;Qiyu Rong;Hongyuan Jing","doi":"10.1109/TIM.2024.3497132","DOIUrl":null,"url":null,"abstract":"Object detection plays a crucial role in the fields of autonomous driving, security surveillance, unmanned aerial vehicle (UAV), and so on. However, the performance of detectors can be drastically degraded by adverse weather conditions, such as fog, rain, and snow. This is because detectors are usually trained on images taken in clear weather conditions but tested under adverse weather conditions. There is a domain shift problem between images captured in adverse weather and those taken in clear weather. In this article, we propose a robust detection framework called two training stage domain adaptation you only look once (TTSDA-YOLO), which performs well in both normal and adverse weather conditions based on YOLOv7. We design a new training strategy that fully utilizes auxiliary domains to transfer knowledge from the source domain to the target domain. This training strategy consists of two stages. In the first training stage, we address the disparity in feature distributions between normal weather images and adverse weather images. We use a multiscale image-level domain adaptation (IDA) module to gradually adapt the normal weather domain to the adverse weather domain. In the second training stage, we make full use of the auxiliary domain by inputting it into the network as a training set. To prevent new domain shifts from being generated during the training process, we design a backbone regularization module (BRM). Extensive experimental results of the proposed TTSDA-YOLO on benchmark datasets show that our approach can significantly improve the detection performance of the network in adverse weather conditions.","PeriodicalId":13341,"journal":{"name":"IEEE Transactions on Instrumentation and Measurement","volume":"74 ","pages":"1-13"},"PeriodicalIF":5.6000,"publicationDate":"2024-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"TTSDA-YOLO: A Two Training Stage Domain Adaptation Framework for Object Detection in Adverse Weather\",\"authors\":\"Mengmeng Zhang;Qiyu Rong;Hongyuan Jing\",\"doi\":\"10.1109/TIM.2024.3497132\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Object detection plays a crucial role in the fields of autonomous driving, security surveillance, unmanned aerial vehicle (UAV), and so on. However, the performance of detectors can be drastically degraded by adverse weather conditions, such as fog, rain, and snow. This is because detectors are usually trained on images taken in clear weather conditions but tested under adverse weather conditions. There is a domain shift problem between images captured in adverse weather and those taken in clear weather. In this article, we propose a robust detection framework called two training stage domain adaptation you only look once (TTSDA-YOLO), which performs well in both normal and adverse weather conditions based on YOLOv7. We design a new training strategy that fully utilizes auxiliary domains to transfer knowledge from the source domain to the target domain. This training strategy consists of two stages. In the first training stage, we address the disparity in feature distributions between normal weather images and adverse weather images. We use a multiscale image-level domain adaptation (IDA) module to gradually adapt the normal weather domain to the adverse weather domain. In the second training stage, we make full use of the auxiliary domain by inputting it into the network as a training set. To prevent new domain shifts from being generated during the training process, we design a backbone regularization module (BRM). Extensive experimental results of the proposed TTSDA-YOLO on benchmark datasets show that our approach can significantly improve the detection performance of the network in adverse weather conditions.\",\"PeriodicalId\":13341,\"journal\":{\"name\":\"IEEE Transactions on Instrumentation and Measurement\",\"volume\":\"74 \",\"pages\":\"1-13\"},\"PeriodicalIF\":5.6000,\"publicationDate\":\"2024-11-13\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Instrumentation and Measurement\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10752537/\",\"RegionNum\":2,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Instrumentation and Measurement","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10752537/","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
TTSDA-YOLO: A Two Training Stage Domain Adaptation Framework for Object Detection in Adverse Weather
Object detection plays a crucial role in the fields of autonomous driving, security surveillance, unmanned aerial vehicle (UAV), and so on. However, the performance of detectors can be drastically degraded by adverse weather conditions, such as fog, rain, and snow. This is because detectors are usually trained on images taken in clear weather conditions but tested under adverse weather conditions. There is a domain shift problem between images captured in adverse weather and those taken in clear weather. In this article, we propose a robust detection framework called two training stage domain adaptation you only look once (TTSDA-YOLO), which performs well in both normal and adverse weather conditions based on YOLOv7. We design a new training strategy that fully utilizes auxiliary domains to transfer knowledge from the source domain to the target domain. This training strategy consists of two stages. In the first training stage, we address the disparity in feature distributions between normal weather images and adverse weather images. We use a multiscale image-level domain adaptation (IDA) module to gradually adapt the normal weather domain to the adverse weather domain. In the second training stage, we make full use of the auxiliary domain by inputting it into the network as a training set. To prevent new domain shifts from being generated during the training process, we design a backbone regularization module (BRM). Extensive experimental results of the proposed TTSDA-YOLO on benchmark datasets show that our approach can significantly improve the detection performance of the network in adverse weather conditions.
期刊介绍:
Papers are sought that address innovative solutions to the development and use of electrical and electronic instruments and equipment to measure, monitor and/or record physical phenomena for the purpose of advancing measurement science, methods, functionality and applications. The scope of these papers may encompass: (1) theory, methodology, and practice of measurement; (2) design, development and evaluation of instrumentation and measurement systems and components used in generating, acquiring, conditioning and processing signals; (3) analysis, representation, display, and preservation of the information obtained from a set of measurements; and (4) scientific and technical support to establishment and maintenance of technical standards in the field of Instrumentation and Measurement.