Haolong Fu;Hanhao Liu;Jin Yuan;Xuan He;Jiacheng Lin;Zhiyong Li
{"title":"YOLO-Adaptor: A Fast Adaptive One-Stage Detector for Non-Aligned Visible-Infrared Object Detection","authors":"Haolong Fu;Hanhao Liu;Jin Yuan;Xuan He;Jiacheng Lin;Zhiyong Li","doi":"10.1109/TIV.2024.3393015","DOIUrl":null,"url":null,"abstract":"Visible-infrared object detection has attracted increasing attention recently due to its superior performance and cost-efficiency. Most existing methods focus on the detection of strictly-aligned data, significantly limiting its practical applications. Although several researchers have attempted to explore weakly-aligned visible-infrared object detection, they are limited to small translational deviations and suffer from a low detection speed. This paper first explores non-aligned visible-infrared object detection with complex deviations in translation, scaling, and rotation, and proposes a fast one-stage detector YOLO-Adaptor, which introduces a lightweight multi-modal adaptor to simultaneously predict alignment parameters and confidence weights between modalities. The adaptor adopts a feature-level alignment during the feature extraction process, ensuring high alignment efficiency. Moreover, we introduce a feature contrastive learning loss to guide the alignment learning of the adaptor, aiming to reduce the representation gap between the two modalities in hyperbolic space to implement feature spatial and distributional consistency. Extensive experiments are conducted on three datasets, including one weakly-aligned and two non-aligned datasets, and the experimental results demonstrate that YOLO-Adaptor could achieve significant performance improvements in terms of speed and accuracy.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 11","pages":"7070-7083"},"PeriodicalIF":14.0000,"publicationDate":"2024-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Intelligent Vehicles","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10508329/","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Visible-infrared object detection has attracted increasing attention recently due to its superior performance and cost-efficiency. Most existing methods focus on the detection of strictly-aligned data, significantly limiting its practical applications. Although several researchers have attempted to explore weakly-aligned visible-infrared object detection, they are limited to small translational deviations and suffer from a low detection speed. This paper first explores non-aligned visible-infrared object detection with complex deviations in translation, scaling, and rotation, and proposes a fast one-stage detector YOLO-Adaptor, which introduces a lightweight multi-modal adaptor to simultaneously predict alignment parameters and confidence weights between modalities. The adaptor adopts a feature-level alignment during the feature extraction process, ensuring high alignment efficiency. Moreover, we introduce a feature contrastive learning loss to guide the alignment learning of the adaptor, aiming to reduce the representation gap between the two modalities in hyperbolic space to implement feature spatial and distributional consistency. Extensive experiments are conducted on three datasets, including one weakly-aligned and two non-aligned datasets, and the experimental results demonstrate that YOLO-Adaptor could achieve significant performance improvements in terms of speed and accuracy.
期刊介绍:
The IEEE Transactions on Intelligent Vehicles (T-IV) is a premier platform for publishing peer-reviewed articles that present innovative research concepts, application results, significant theoretical findings, and application case studies in the field of intelligent vehicles. With a particular emphasis on automated vehicles within roadway environments, T-IV aims to raise awareness of pressing research and application challenges.
Our focus is on providing critical information to the intelligent vehicle community, serving as a dissemination vehicle for IEEE ITS Society members and others interested in learning about the state-of-the-art developments and progress in research and applications related to intelligent vehicles. Join us in advancing knowledge and innovation in this dynamic field.