TS-DETR: Multi-scale DETR for traffic sign detection and recognition

Abstract

Traffic Sign Detection and Recognition (TSDR) is essential for driverless cars and driver assistance systems. Machine vision tasks, such as TSDR, have gained significant attention. Convolutional neural networks (CNNs) are often employed for these tasks, but the introduction of visual transformers offers an alternative approach to global feature learning. As a novel object detection paradigm, DETR (Detection Transformer) can correlate contextual information, making it suitable for TSDR tasks. In this paper, we propose a novel network, TS-DETR, designed for TSDR. This network builds upon the RT-DETR (Real-Time Detection Transformer) and incorporates a multi-sequence feature fusion strategy along with an enhanced attention mechanism. First, we design a feature connection module alongside a multi-sequence feature fusion module, allowing the TS-DETR network to acquire more comprehensive multi-scale information. Then, considering the positional characteristics of traffic signs, we introduce channel and spatial attention modules, which enhance the network’s ability to utilize this information. Finally, we incorporate an inverted residual moving block to balance network performance. The experimental outcomes on the publicly available CCTSDB dataset illustrate that TS-DETR improves detection accuracy for all three traffic sign categories, outperforms YOLOv5s and YOLOv8s, and improves the recognition rate by 3.5% compared to the original RT-DETR algorithm. These results underscore the effectiveness of TS-DETR for TSDR tasks.