Self-Supervised Multi-Granularity Graph Attention Network for Vision-Based Driver Fatigue Detection

Driver fatigue is one of the main causes of traffic accidents. Current vision-based methods for detecting driver fatigue lack robustness in the presence of interfering images, and exhibit insufficient ability to focus on frames containing crucial information. To address these issues, we propose a Self-supervised Multi-granularity Graph Attention Network (SMGA-Net) for driver fatigue detection. The network mainly contains the following contributions: Firstly, with the multi-task self-supervised learning strategy, a novel method called Image Restoration based Self-supervised Learning (IRS-Learning) is proposed to enhance the network's robustness when processing interfering images. Secondly, with the graph attention mechanism, a Multi-head Graph Attention (MG-Attention) module is designed to concentrate on frames that contain crucial information by assigning importance weights to each frame. In addition, a Cross Attention Feature Fusion (CAF-Fusion) method is proposed to adaptively merge the multi-granularity features and emphasize effective information contained therein. Experiments performed on the National TsingHua University Drowsy Driver Detection (NTHU-DDD) dataset show that the proposed SMGA-Net based driver fatigue detection method outperforms the state-of-art methods.