Dual-perspective safety driver secondary task detection method based on swin-transformer and cross-attention

Abstract

With the rapid development of autonomous driving technology, the safety of autonomous vehicles has attracted widespread attention. However, existing research primarily focuses on traditional driver behavior detection and typically adopts a single perspective for analysis, lacking a comprehensive study of the safety driver’s status in autonomous driving scenarios. Therefore, this study proposes an autonomous driving safety driver secondary task detection method (ST-CAFL) based on dual perspectives using Swin-Transformer (Swin-T) and cross-attention (CA) to improve the accuracy and robustness of secondary task detection. Firstly, the ST-CAFL framework efficiently captures multi-scale spatial information through Swin-T’s hierarchical feature extraction mechanism. Secondly, the CA mechanism effectively integrates information from dual perspectives, providing a more comprehensive capture of the safety driver’s behavioral characteristics. Additionally, this study introduces center loss to improve classification accuracy and enhance the model’s ability to recognize secondary tasks performed by the safety driver, thereby achieving more comprehensive and accurate detection. To evaluate the effectiveness of the proposed method, we created a dual-perspective safety driver behavior detection (ASD) dataset and conducted extensive experiments on this dataset. The results indicate that the ST-CAFL framework achieved an accuracy of 96.15% on the SAM dataset and 94.39% on the ASD dataset. To further validate the applicability of this method, we conducted extensive experiments across both datasets. Moreover, ST-CAFL outperformed several existing methods in terms of detection performance in comparative evaluations. This research fills the gap in the autonomous driving safety driver secondary task detection field and provides an essential reference for the design and implementation of future autonomous driving systems, possessing broad application value and research significance.