ETR: Enhancing Taillight Recognition via Transformer-Based Video Classification

Abstract

In autonomous driving, efficiently and accurately recognizing taillight states using dashcams is essential for interpreting other vehicles’ intentions. Recent video-based taillight recognition methods outperform earlier image-based approaches. However, they face challenges in efficiently integrating spatiotemporal information and managing high computational costs. In this paper, we introduce ETR, an accurate and efficient Transformer-based video classification model designed to enhance taillight recognition. Specifically, we first design a lightweight backbone to extract temporal and spatial features from videos and generate queries with prior information. Next, we develop a hierarchical progressive Transformer decoder that integrates feature maps from different levels of the backbone to enhance the model’s global information. Finally, we employ a classification head to predict the taillight state of the video. Additionally, we introduce a public dataset, ETR-Taillights, to address the current lack of open datasets for vehicle taillight recognition. The dataset contains 28,799 dashcam video clips, making it the largest public taillight recognition dataset. Experiments show that our method achieves a 91.69% F-measure on the ETR-Taillights dataset, surpassing the latest taillight recognition methods, VIF by 6.94% and CNN-LSTM by 10.82%. Additionally, it achieves an inference speed of 45.06 FPS, being 3.6 times faster than VIF. Furthermore, we conduct real-world road tests to demonstrate our method’s robustness and effectiveness in practical scenarios. Our model and dataset are available at https://github.com/yang590/vehicle-taillight.